Logging, Debugging & Error Handling

Published by Brayan K

When your codebase is tiny, print() and random try/except blocks "work". Once you have APIs, background workers, cron jobs, queues, and multiple servers, that approach falls apart. Learn professional-grade debugging + logging + error-handling systems used by real engineering teams.

Part of the free Python course at LearnCodingFast — hands-on lessons with examples you run in your browser, plus practice exercises and a quick quiz.

Standard output and standard error

This lesson starts where Exception Handling stops. That lesson catches an error and decides what happens next; this one records it, with Python's logging module, so you can read it later, and then shows how to stop a running program and look inside it with Python's debugger, pdb. Exception classes of your own and raise … from belong to Custom Exceptions & Exception Chaining, and warnings to The warnings Module.

A program has two streams for text. print() writes to standard output. Standard error is the second stream, meant for problems and reports, and it's where logging writes unless you tell it otherwise. A terminal shows both, so they look like one stream until you separate them. On Linux and macOS, these two commands do:

python app.py 2>errors.log
python app.py >/dev/null

With an app.py that prints report ready and then calls logging.warning("disk almost full"), the first command shows report ready and puts WARNING:root:disk almost full in errors.log; the second throws standard output away and shows only the warning. A logger you make yourself writes to standard error too:

import logging

log = logging.getLogger("shop")
print("report ready")
log.info("3 orders read")
log.warning("disk almost full")
print("2 files written")

# ✅ Expected output:
# report ready
# 2 files written

In a terminal a third line appears between those two: disk almost full, on standard error. 3 orders read appears nowhere. Until logging is set up, a logger passes on only WARNING and above, and they come out through Python's last-resort handler, which writes the message alone to standard error. This site's editor shows such a line after everything printed, in yellow and marked ⚠️. The panels show standard output, so the examples from here on send their log lines there.

Every logger reports to one at the top, the root logger. Before anything is set up, its level is WARNING and it has no handlers, the objects that write lines somewhere:

import logging

root = logging.getLogger()
print(root.level, logging.getLevelName(root.level))
print(len(root.handlers))

# ✅ Expected output:
# 30 WARNING
# 0

The module's own functions, such as logging.warning("disk almost full"), set the root logger up the first time one is called: a handler on standard error, with the format %(levelname)s:%(name)s:%(message)s. That's where WARNING:root:disk almost full above came from. The examples below set logging up themselves instead.

Set logging up once, with basicConfig()

logging.basicConfig() sets up the root logger: the lowest level to show, the format of each line, and where the lines go. A program does it once, where it starts:

import logging
import sys

logging.basicConfig(stream=sys.stdout, level=logging.INFO, force=True,
                    format="%(levelname)s %(name)s: %(message)s")
log = logging.getLogger("shop")
log.debug("basket has 3 items")
log.info("order 7 received")
log.warning("card declined for order 7")

# ✅ Expected output:
# INFO shop: order 7 received
# WARNING shop: card declined for order 7

The DEBUG line is below the INFO level, so it's dropped. Two of the arguments are there for this site. stream=sys.stdout sends the lines to standard output, so the panel can show them; a real program would usually leave them on standard error. force=True replaces any set-up that's already there: Python keeps its loggers for as long as the program runs, and this site's editor keeps one Python for every Run on a page, so without it a second Run would print nothing. A program run in a terminal starts fresh each time and doesn't need it.

Without force=True, a second call does nothing at all:

import logging
import sys

logging.basicConfig(stream=sys.stdout, level=logging.WARNING, force=True,
                    format="%(levelname)s %(name)s: %(message)s")
logging.basicConfig(stream=sys.stdout, level=logging.DEBUG,
                    format="%(message)s")
log = logging.getLogger("shop")
log.debug("basket has 3 items")
log.warning("card declined")

# ✅ Expected output:
# WARNING shop: card declined

Its documentation says so: it does nothing "if the root logger already has handlers configured, unless the keyword argument force is set to True". The DEBUG line stays hidden, and the format is still the first one. With force=True the second call wins:

import logging
import sys

logging.basicConfig(stream=sys.stdout, level=logging.WARNING, force=True,
                    format="%(levelname)s %(name)s: %(message)s")
logging.basicConfig(stream=sys.stdout, level=logging.DEBUG, force=True,
                    format="%(message)s")
log = logging.getLogger("shop")
log.debug("basket has 3 items")
log.warning("card declined")

# ✅ Expected output:
# basket has 3 items
# card declined

The usual way to fall into this: a module logs something as it's imported, before the program reaches its basicConfig().

import logging
import sys

logging.info("loading settings")
logging.basicConfig(stream=sys.stdout, level=logging.DEBUG,
                    format="%(levelname)s %(name)s: %(message)s")
logging.info("settings loaded")
root = logging.getLogger()
print(len(root.handlers), logging.getLevelName(root.level))

# ✅ Expected output:
# 1 WARNING

The first logging.info() set the root logger up, with one handler on standard error and the level WARNING. So the later basicConfig() was ignored, and neither INFO line was shown.

Loggers have names, and the dots make a tree

logging.getLogger() returns the same logger every time it's given the same name, as Design Patterns in Python (Singleton, Factory, Strategy) pointed out. So each part of a program asks for its logger by name instead of having one passed to it:

import logging

a = logging.getLogger("shop")
b = logging.getLogger("shop")
c = logging.getLogger("admin")
print(a is b, a is c)
print(a.name, c.name)

# ✅ Expected output:
# True False
# shop admin

The usual name is the module's own, __name__. In the program you run that's "__main__", as Modules and Packages showed, and in a module of a package it's the module's dotted name:

from pathlib import Path

Path("shop").mkdir(exist_ok=True)
Path("shop/__init__.py").write_text("", encoding="utf-8")
Path("shop/checkout.py").write_text(
    "import logging\n"
    "\n"
    "log = logging.getLogger(__name__)\n"
    "\n"
    "def pay(order):\n"
    "    log.info('order %s paid', order)\n",
    encoding="utf-8",
)

import logging
import sys

logging.basicConfig(stream=sys.stdout, level=logging.INFO, force=True,
                    format="%(levelname)s %(name)s: %(message)s")
import shop.checkout

shop.checkout.pay(7)

# ✅ Expected output:
# INFO shop.checkout: order 7 paid

The dots make a tree: shop.checkout is a child of shop, and every logger is a descendant of the root. A line goes up that tree, and each handler on the way writes it. Neither of these loggers has a handler, so the root's writes both lines:

import logging
import sys

logging.basicConfig(stream=sys.stdout, level=logging.INFO, force=True,
                    format="%(levelname)s %(name)s: %(message)s")
logging.getLogger("shop.checkout").info("order 7 paid")
logging.getLogger("shop.stock").warning("only 2 left")

# ✅ Expected output:
# INFO shop.checkout: order 7 paid
# WARNING shop.stock: only 2 left

So a handler on shop and another on the root write each of its lines twice:

import logging
import sys

logging.basicConfig(stream=sys.stdout, level=logging.INFO, force=True,
                    format="root's handler: %(message)s")
shop = logging.getLogger("shop")
shop.handlers.clear()
handler = logging.StreamHandler(sys.stdout)
handler.setFormatter(logging.Formatter("shop's handler: %(message)s"))
shop.addHandler(handler)

logging.getLogger("shop.checkout").info("order 7 paid")

# ✅ Expected output:
# shop's handler: order 7 paid
# root's handler: order 7 paid

propagate = False stops a line at that logger, so the root's handler never sees it. (shop.handlers.clear() is there for this site's editor, which keeps loggers between Runs: without it, every Run would add one more handler.)

import logging
import sys

logging.basicConfig(stream=sys.stdout, level=logging.INFO, force=True,
                    format="root's handler: %(message)s")
shop = logging.getLogger("shop")
shop.handlers.clear()
handler = logging.StreamHandler(sys.stdout)
handler.setFormatter(logging.Formatter("shop's handler: %(message)s"))
shop.addHandler(handler)
shop.propagate = False

logging.getLogger("shop.checkout").info("order 7 paid")

# ✅ Expected output:
# shop's handler: order 7 paid

A set-up function that runs twice does the same thing, one handler per call:

import logging
import sys

def setup():
    log = logging.getLogger("shop")
    log.setLevel(logging.INFO)
    handler = logging.StreamHandler(sys.stdout)
    handler.setFormatter(logging.Formatter("%(levelname)s %(name)s: %(message)s"))
    log.addHandler(handler)
    return log

logging.getLogger("shop").handlers.clear()
setup()
log = setup()
log.info("order 7 paid")
print(len(log.handlers), "handlers")

# ✅ Expected output:
# INFO shop: order 7 paid
# INFO shop: order 7 paid
# 2 handlers

So set logging up once, where the program starts, and let each module only ask for its logger. A library goes further and sets nothing up, leaving that to the program that uses it; the most it adds is logging.NullHandler(), a handler that writes nothing and keeps the last-resort handler quiet.

Levels: two checks decide whether a line appears

Each level is a number: DEBUG 10, INFO 20, WARNING 30, ERROR 40 and CRITICAL 50. A line is shown when its level passes two checks, the logger's level first and then the level of each handler that would write it. A logger with no level of its own, 0, uses its parent's, and so on up to the root:

import logging

logging.getLogger("shop").setLevel(logging.DEBUG)
db = logging.getLogger("shop.db")
print(db.level, db.getEffectiveLevel())
print(logging.getLogger("admin").getEffectiveLevel())

# ✅ Expected output:
# 0 10
# 30

getEffectiveLevel() says which level applies. That's how to quiet one noisy part of a program, by giving its logger a higher level:

import logging
import sys

logging.basicConfig(stream=sys.stdout, level=logging.DEBUG, force=True,
                    format="%(levelname)s %(name)s: %(message)s")
logging.getLogger("shop.db").setLevel(logging.WARNING)

logging.getLogger("shop.db").debug("SELECT * FROM orders")
logging.getLogger("shop.db").warning("slow query")
logging.getLogger("shop.checkout").debug("basket has 3 items")

# ✅ Expected output:
# WARNING shop.db: slow query
# DEBUG shop.checkout: basket has 3 items

A handler's level can only drop more lines. This handler takes DEBUG, but shop has no level and the root's is WARNING, so the DEBUG line never gets as far as the handler:

import logging
import sys

log = logging.getLogger("shop")
log.handlers.clear()
log.setLevel(logging.NOTSET)
log.propagate = False
handler = logging.StreamHandler(sys.stdout)
handler.setLevel(logging.DEBUG)
log.addHandler(handler)

log.debug("basket has 3 items")
log.warning("card declined")

# ✅ Expected output:
# card declined

A handler at ERROR next to one that takes everything sends the serious lines to a second place as well; the practice exercise below builds one. Levels can also be given by name, in capitals: log.setLevel("INFO") works. In lower case it stops the program:

import logging

log = logging.getLogger("shop")
log.setLevel("info")

# ✅ Expected error:
# ValueError: Unknown level: 'info'

Give values as arguments, not in an f-string

f-strings & String Formatting mentioned that % formatting still turns up in logging calls. This is why: a logging call takes a message with %s placeholders and the values after it, and fills them in only for a line it's going to show.

import logging
import sys

logging.basicConfig(stream=sys.stdout, level=logging.INFO, force=True,
                    format="%(levelname)s %(name)s: %(message)s")
log = logging.getLogger("shop")

class Basket:
    def __str__(self):
        print("  (making the basket's text)")
        return "3 items"

basket = Basket()
log.debug("basket: %s", basket)
log.info("basket: %s", basket)

# ✅ Expected output:
#   (making the basket's text)
# INFO shop: basket: 3 items

The basket's text was made once, for the INFO line; the DEBUG line was dropped before anything was filled in. An f-string is built before the call, so it's made for the hidden line too:

import logging
import sys

logging.basicConfig(stream=sys.stdout, level=logging.INFO, force=True,
                    format="%(levelname)s %(name)s: %(message)s")
log = logging.getLogger("shop")

class Basket:
    def __str__(self):
        print("  (making the basket's text)")
        return "3 items"

basket = Basket()
log.debug(f"basket: {basket}")
log.info(f"basket: {basket}")

# ✅ Expected output:
#   (making the basket's text)
#   (making the basket's text)
# INFO shop: basket: 3 items

The values themselves are still worked out, though: log.debug("items: %s", count_items()) calls count_items() whether the line is shown or not. For something slow, check first, with if log.isEnabledFor(logging.DEBUG):.

The comma that works in print() doesn't work here:

import logging
import sys

logging.basicConfig(stream=sys.stdout, level=logging.INFO, force=True,
                    format="%(levelname)s %(name)s: %(message)s")
log = logging.getLogger("shop")
total = 12.5
log.info("Total:", total)
print("still running")

# ✅ Expected output:
# still running

12.5 is a value with no %s to go into, so formatting fails inside logging, and logging doesn't stop the program over it. The line is lost, and a report goes to standard error instead. It starts --- Logging error ---, names TypeError: not all arguments converted during string formatting, lists the calls that led there, and ends with Message: 'Total:' and Arguments: (12.5,).

With no values after it, a message is written as it is, so a % sign is fine. With values, the message is a % format, and a percent sign is written %%:

import logging
import sys

logging.basicConfig(stream=sys.stdout, level=logging.INFO, force=True,
                    format="%(levelname)s %(name)s: %(message)s")
log = logging.getLogger("shop")
log.info("50% done")
log.info("50%% done for order %s", 7)

# ✅ Expected output:
# INFO shop: 50% done
# INFO shop: 50% done for order 7

The format decides what each line says

A format names the fields each line shows, each written %(name)s, or %(name)d for a number. -8 pads a field to eight characters, and funcName and lineno say where the call was made:

import logging
import sys

logging.basicConfig(stream=sys.stdout, level=logging.INFO, force=True,
                    format="%(levelname)-8s %(name)s %(funcName)s:%(lineno)d %(message)s")

def charge(order):
    logging.getLogger("shop").warning("card declined for %s", order)

charge("A-7")

# ✅ Expected output:
# WARNING  shop charge:8 card declined for A-7

A format written with braces, as in str.format(), needs the argument style="{" as well. Without it, the set-up stops:

import logging
import sys

logging.basicConfig(stream=sys.stdout, level=logging.INFO, force=True,
                    format="{levelname} {name}: {message}")
logging.getLogger("shop").info("order 7 received")

# ✅ Expected error:
# ValueError: Invalid format '{levelname} {name}: {message}' for '%' style

extra= adds fields of your own to a line's record, and the format shows them by name:

import logging
import sys

logging.basicConfig(stream=sys.stdout, level=logging.INFO, force=True,
                    format="%(levelname)s user=%(user)s %(message)s")
log = logging.getLogger("shop")
log.warning("card declined", extra={"user": "ada"})

# ✅ Expected output:
# WARNING user=ada card declined

A field the format doesn't name is on the record but not in the line, so the default format leaves user_id out here. (A key the record already has, such as message, stops the call with KeyError: "Attempt to overwrite 'message' in LogRecord".)

import logging
import sys

logging.basicConfig(stream=sys.stdout, level=logging.INFO, force=True)
logging.getLogger("web").info("user logged in", extra={"user_id": 7})

# ✅ Expected output:
# INFO:web:user logged in

Programs that collect log lines can search them by field when each line is JSON. A formatter of your own decides the whole line; this one writes one JSON object per line with json.dumps(), as Working with JSON showed:

import json
import logging
import sys

class JsonLines(logging.Formatter):
    def format(self, record):
        entry = {"level": record.levelname, "logger": record.name,
                 "message": record.getMessage()}
        entry.update(getattr(record, "fields", {}))
        return json.dumps(entry)

handler = logging.StreamHandler(sys.stdout)
handler.setFormatter(JsonLines())
logging.basicConfig(level=logging.INFO, handlers=[handler], force=True)

log = logging.getLogger("web")
log.info("user logged in", extra={"fields": {"user_id": 7, "ip": "10.0.0.5"}})
log.warning("card declined")

# ✅ Expected output:
# {"level": "INFO", "logger": "web", "message": "user logged in", "user_id": 7, "ip": "10.0.0.5"}
# {"level": "WARNING", "logger": "web", "message": "card declined"}

A filter on a handler sees every line that handler writes, and can add a field to each one, such as the id of the request being handled:

import logging
import sys

class AddRequestId(logging.Filter):
    def __init__(self, request_id):
        super().__init__()
        self.request_id = request_id

    def filter(self, record):
        record.request_id = self.request_id
        return True

handler = logging.StreamHandler(sys.stdout)
handler.setFormatter(logging.Formatter("[%(request_id)s] %(name)s: %(message)s"))
handler.addFilter(AddRequestId("r-1"))
logging.basicConfig(level=logging.INFO, handlers=[handler], force=True)
logging.getLogger("web").info("GET /checkout")
logging.getLogger("web.payments").warning("card declined")

# ✅ Expected output:
# [r-1] web: GET /checkout
# [r-1] web.payments: card declined

The filter is on the handler for a reason. A filter on a logger sees only that logger's own calls, not its children's: on web, it would miss web.payments' line, which would then fail with a --- Logging error --- report, since its format names a field that line doesn't have.

Some fields change from run to run, so no panel here shows them: %(asctime)s, the date and time of the line, and %(process)d, the number of the running process. A real format usually starts with %(asctime)s.

Recording an error you've caught

Inside an except block, log.exception("charge failed") logs an ERROR line followed by the whole traceback, the report Python prints for an error nothing catches. A traceback's middle lines name files and line numbers, and they differ between computers, Python versions and this site's editor, so the examples here use a formatter that keeps only its last line:

import logging
import sys
import traceback

class ShortErrors(logging.Formatter):
    def formatException(self, exc_info):
        return "  " + traceback.format_exception_only(exc_info[1])[-1].strip()

handler = logging.StreamHandler(sys.stdout)
handler.setFormatter(ShortErrors("%(levelname)s %(name)s: %(message)s"))
logging.basicConfig(level=logging.INFO, handlers=[handler], force=True)
log = logging.getLogger("shop")

def charge(amount):
    if amount > 1000:
        raise ValueError("amount over limit")
    return amount

try:
    charge(5000)
except ValueError:
    log.exception("charge failed")
print("still running")

# ✅ Expected output:
# ERROR shop: charge failed
#   ValueError: amount over limit
# still running

With the usual formatter, the same call writes ERROR shop: charge failed, then the traceback from Traceback (most recent call last): down to ValueError: amount over limit, and the program carries on. log.error() with the error in the message records what went wrong, but not where:

import logging
import sys

logging.basicConfig(stream=sys.stdout, level=logging.INFO, force=True,
                    format="%(levelname)s %(name)s: %(message)s")
log = logging.getLogger("shop")

def charge(amount):
    if amount > 1000:
        raise ValueError("amount over limit")
    return amount

try:
    charge(5000)
except ValueError as e:
    log.error("charge failed: %s", e)

# ✅ Expected output:
# ERROR shop: charge failed: amount over limit

log.exception() is log.error() with exc_info=True: it logs at ERROR and adds the error being handled. Outside an except block there's none to add:

import logging
import sys

logging.basicConfig(stream=sys.stdout, level=logging.INFO, force=True,
                    format="%(levelname)s %(name)s: %(message)s")
log = logging.getLogger("shop")
log.error("charge failed", exc_info=True)

# ✅ Expected output:
# ERROR shop: charge failed
# NoneType: None

Logging an error where it's caught and then re-raising it writes the same failure twice:

import logging
import sys

logging.basicConfig(stream=sys.stdout, level=logging.INFO, force=True,
                    format="%(levelname)s %(name)s: %(message)s")
log = logging.getLogger("shop")

def charge(amount):
    if amount > 1000:
        raise ValueError("amount over limit")
    return amount

def checkout(amount):
    try:
        return charge(amount)
    except ValueError:
        log.error("checkout failed")
        raise

try:
    checkout(5000)
except ValueError:
    log.error("request failed")

# ✅ Expected output:
# ERROR shop: checkout failed
# ERROR shop: request failed

Log an error once, in the place that handles it. A program that runs for a long time, such as a batch job or a server, logs the error and carries on with the next piece of work, so one bad row costs one line in the log rather than the whole run:

import logging
import sys
import traceback

class ShortErrors(logging.Formatter):
    def formatException(self, exc_info):
        return "  " + traceback.format_exception_only(exc_info[1])[-1].strip()

handler = logging.StreamHandler(sys.stdout)
handler.setFormatter(ShortErrors("%(levelname)s %(name)s: %(message)s"))
logging.basicConfig(level=logging.INFO, handlers=[handler], force=True)
log = logging.getLogger("import")

rows = [("A-1", "2"), ("A-2", "two"), ("A-3", "5")]
imported = 0
for order, qty in rows:
    try:
        count = int(qty)
    except ValueError:
        log.exception("order %s skipped", order)
        continue
    imported += 1
    log.info("order %s: %d items", order, count)
log.info("%d of %d orders imported", imported, len(rows))

# ✅ Expected output:
# INFO import: order A-1: 2 items
# ERROR import: order A-2 skipped
#   ValueError: invalid literal for int() with base 10: 'two'
# INFO import: order A-3: 5 items
# INFO import: 2 of 3 orders imported

An error nothing catches can be logged too. Just before such an error stops the program, Python calls sys.excepthook with it:

import logging
import sys

logging.basicConfig(stream=sys.stdout, level=logging.INFO, force=True,
                    format="%(levelname)s %(name)s: %(message)s")
log = logging.getLogger("shop")

def log_uncaught(kind, error, tb):
    log.critical("uncaught %s: %s", kind.__name__, error)

sys.excepthook = log_uncaught
print("starting")
int("ten")

Run with python3, this prints starting and then CRITICAL shop: uncaught ValueError: invalid literal for int() with base 10: 'ten' instead of the usual report, and the program ends with exit status 1, as any failed program does. This site's editor shows its own report of the error and doesn't call the hook.

Reading a traceback in code: the traceback module

Introduction to Python read a traceback from the bottom up, and Exception Handling's traceback.print_exc() prints the report and lets the program carry on; it writes to standard error, as logging does. The traceback module also hands you the report's parts. extract_tb() lists the calls that led to the error, from the outside in:

import traceback

def charge(amount):
    if amount > 1000:
        raise ValueError("amount over limit")
    return amount

def checkout(amount):
    return charge(amount)

try:
    checkout(5000)
except ValueError as e:
    for frame in traceback.extract_tb(e.__traceback__):
        print(frame.name, "line", frame.lineno)
    print(traceback.format_exception_only(e))

# ✅ Expected output:
# <module> line 12
# checkout line 9
# charge line 5
# ['ValueError: amount over limit\n']

Line 12 is the call at the bottom of the program, line 9 the call inside checkout(), and line 5 the raise. format_exc() returns the whole report as text, the same text print_exc() prints:

import traceback

def charge(amount):
    if amount > 1000:
        raise ValueError("amount over limit")
    return amount

try:
    charge(5000)
except ValueError:
    report = traceback.format_exc()

lines = report.splitlines()
print(lines[0])
print(lines[-1])

# ✅ Expected output:
# Traceback (most recent call last):
# ValueError: amount over limit

An error raised while another is being handled gives a report in two parts, joined by a sentence that says how they're related:

import traceback

prices = {"tea": 2}
try:
    try:
        prices["coffee"]
    except KeyError:
        int("ten")
except ValueError:
    for line in traceback.format_exc().splitlines():
        if not line.startswith(" "):
            print(line)

# ✅ Expected output:
# Traceback (most recent call last):
# KeyError: 'coffee'
#
# During handling of the above exception, another exception occurred:
#
# Traceback (most recent call last):
# ValueError: invalid literal for int() with base 10: 'ten'

raise … from joins them with a different sentence, "The above exception was the direct cause of the following exception:". Custom Exceptions & Exception Chaining covers both.

Log files

A FileHandler writes lines to a file, and the file can be read back like any other. (This site's editor empties its folder before each Run, so the file starts empty every time.)

import logging
from pathlib import Path

handler = logging.FileHandler("app.log", encoding="utf-8")
handler.setFormatter(logging.Formatter("%(levelname)s %(name)s: %(message)s"))
logging.basicConfig(level=logging.INFO, handlers=[handler], force=True)

log = logging.getLogger("shop")
log.info("order 7 received")
log.warning("card declined for order 7")
handler.close()
print(Path("app.log").read_text(encoding="utf-8"), end="")

# ✅ Expected output:
# INFO shop: order 7 received
# WARNING shop: card declined for order 7

A log file grows for as long as the program runs. RotatingFileHandler starts a new file when the current one would pass maxBytes, keeps backupCount old ones, and deletes the oldest. The limit here is tiny, so every line gets a file of its own:

import logging
from logging.handlers import RotatingFileHandler
from pathlib import Path

handler = RotatingFileHandler("app.log", maxBytes=40, backupCount=2, encoding="utf-8")
handler.setFormatter(logging.Formatter("%(message)s"))
log = logging.getLogger("shop")
log.handlers.clear()
log.addHandler(handler)
log.propagate = False
for n in range(1, 8):
    log.warning("order %d is waiting for payment", n)
handler.close()

for path in sorted(Path(".").glob("app.log*")):
    print(path.name, "|", path.read_text(encoding="utf-8").strip())

# ✅ Expected output:
# app.log | order 7 is waiting for payment
# app.log.1 | order 6 is waiting for payment
# app.log.2 | order 5 is waiting for payment

For a bigger set-up, logging.config.dictConfig() takes all of it as one dictionary: formats, handlers, and a level for each logger.

import logging
import logging.config

logging.config.dictConfig({
    "version": 1,
    "disable_existing_loggers": False,
    "formatters": {"short": {"format": "%(levelname)s %(name)s: %(message)s"}},
    "handlers": {"console": {"class": "logging.StreamHandler", "formatter": "short",
                             "stream": "ext://sys.stdout"}},
    "loggers": {"shop.db": {"level": "WARNING"}},
    "root": {"level": "DEBUG", "handlers": ["console"]},
})
logging.getLogger("shop.checkout").debug("basket has 3 items")
logging.getLogger("shop.db").debug("SELECT * FROM orders")
logging.getLogger("shop.db").warning("slow query")

# ✅ Expected output:
# DEBUG shop.checkout: basket has 3 items
# WARNING shop.db: slow query

"disable_existing_loggers": False keeps existing loggers enabled. With the default True, existing non-root loggers are disabled unless they or their ancestors are named in the configuration.

Debugging with pdb

A log says what a program did. A debugger stops it at a line you choose so you can look at its variables there. Python's is pdb, and it needs a terminal to type its commands into, so this part is for a terminal rather than this site's editor. Take this checkout.py:

def total(prices):
    result = 0
    for price in prices:
        result += price
    return result

print(total([2, 3]))

python -m pdb checkout.py starts it under pdb, stopped before its first line, with a (Pdb) prompt waiting for a command. These commands, typed one at a time, give the same answers on Python 3.10–3.14:

next runs one line, step goes into the function a line calls, and where shows the calls that led to the current line. breakpoint() written in your code stops there when the program is run normally. Where exactly differs: Python 3.10–3.12 stop on the line after it, and 3.13 and later on the breakpoint() line itself. python -m pdb -c continue crash.py runs a program until it fails, then stops at the failing line with its variables still there to print.

breakpoint() calls sys.breakpointhook(), which starts pdb, and a program can replace it:

import sys

def report(*args, **kwargs):
    print("breakpoint() called sys.breakpointhook")

sys.breakpointhook = report
print("before")
breakpoint()
print("after")

# ✅ Expected output:
# before
# breakpoint() called sys.breakpointhook
# after

With Python's default sys.breakpointhook, setting PYTHONBREAKPOINT to 0 disables breakpoint(). A custom hook, such as the preceding example's, decides its own behavior and need not read that environment variable. Start a fresh editor or press Reset before trying the default-hook example. In a terminal it's PYTHONBREAKPOINT=0 python app.py; from inside the program:

import os

os.environ["PYTHONBREAKPOINT"] = "0"
print("before")
breakpoint()
print("after")

# ✅ Expected output:
# before
# after

assert: a check that can be switched off

assert checks something that should always be true at that point, and stops the program with an AssertionError, and your message, when it isn't:

def add_line(total, price):
    assert price >= 0, f"price {price} is negative"
    return total + price

print(add_line(10, 5))
print(add_line(10, -5))

# ✅ Expected output:
# 15

# ✅ Expected error:
# AssertionError: price -5 is negative

Running a program with python -O removes every assert from it: the same function then returns 5 for the bad price, without a word. So an assert catches your own mistakes while you work, and a check on data from outside, such as what a user typed, needs an if and an error that's always raised. Tests use assert too, in Testing with pytest: Fixtures, Parametrisation & Mocks.

Practice: send errors to a second handler

The logger shop should write every line, from DEBUG up, through the handler everything, and the errors once more through alerts. Fill in the two levels: the logger's own, so nothing is dropped before the handlers, and the alerts handler's.

import logging
import sys

log = logging.getLogger("shop")
log.handlers.clear()
log.setLevel(logging.___)
log.propagate = False

everything = logging.StreamHandler(sys.stdout)
everything.setFormatter(logging.Formatter("log:   %(levelname)s %(message)s"))
alerts = logging.StreamHandler(sys.stdout)
alerts.setLevel(logging.___)
alerts.setFormatter(logging.Formatter("ALERT: %(levelname)s %(message)s"))
log.addHandler(everything)
log.addHandler(alerts)

log.debug("basket has 3 items")
log.info("order 7 paid")
log.error("payment service down")

# 👉 replace ___ with DEBUG
# 👉 replace ___ with ERROR

# ✅ Expected output:
# log:   DEBUG basket has 3 items
# log:   INFO order 7 paid
# log:   ERROR payment service down
# ALERT: ERROR payment service down

On your own: an order import that logs

Write the import for this list of orders:

import logging
import sys

orders = [("A-1", "2"), ("A-2", "two"), ("A-3", "5"), ("A-4", "-1")]

# your code here
import logging
import sys

logging.basicConfig(stream=sys.stdout, level=logging.INFO, force=True,
                    format="%(levelname)s %(name)s: %(message)s")
log = logging.getLogger("orders.import")

orders = [("A-1", "2"), ("A-2", "two"), ("A-3", "5"), ("A-4", "-1")]

imported = 0
for order, qty in orders:
    try:
        count = int(qty)
    except ValueError:
        log.warning("order %s skipped: quantity %r is not a number", order, qty)
        continue
    if count <= 0:
        log.error("order %s skipped: quantity %d", order, count)
        continue
    imported += 1
    log.info("order %s: %d items", order, count)

log.info("%d of %d orders imported", imported, len(orders))

# ✅ Expected output:
# INFO orders.import: order A-1: 2 items
# WARNING orders.import: order A-2 skipped: quantity 'two' is not a number
# INFO orders.import: order A-3: 5 items
# ERROR orders.import: order A-4 skipped: quantity -1
# INFO orders.import: 2 of 4 orders imported

Practice quiz

Nothing is set up. A program runs logging.info("starting") and then logging.warning("low disk"). What does the terminal show?

  • INFO:root:starting, then WARNING:root:low disk
  • WARNING:root:low disk
  • low disk
  • Nothing

Answer: WARNING:root:low disk. The first call sets the root logger up with a handler on standard error and the format levelname:name:message. The root's level is WARNING, so only the second line is shown.

logging.basicConfig(stream=sys.stdout, level=logging.WARNING, force=True), then logging.basicConfig(stream=sys.stdout, level=logging.DEBUG), then logging.debug("checking stock") and logging.warning("low stock"). What prints?

  • DEBUG:root:checking stock, then WARNING:root:low stock
  • low stock
  • WARNING:root:low stock
  • It stops with ValueError

Answer: WARNING:root:low stock. The second basicConfig() does nothing, because the root logger already has a handler; only force=True would replace it.

basicConfig(stream=sys.stdout, format="%(message)s", level=logging.INFO, force=True). The logger "shop" also gets its own StreamHandler(sys.stdout). Then logging.getLogger("shop.checkout").info("paid"). What prints?

  • paid
  • Nothing: shop.checkout has no handler
  • paid, then INFO:shop.checkout:paid
  • paid, twice

Answer: paid, twice. The record goes up the tree to every handler on the way: shop's handler writes it, then the root's handler writes it again.

Logging is set up to print INFO lines. A program runs logging.getLogger("shop").info("Total:", 12.5) and then print("done"). What happens?

  • done is printed, and a report starting --- Logging error --- goes to standard error
  • Total: 12.5, then done
  • It stops with TypeError: not all arguments converted during string formatting
  • Total:, then done

Answer: done is printed, and a report starting --- Logging error --- goes to standard error. Extra values fill %s placeholders. With none in the message, formatting fails inside logging, which reports it on standard error and lets the program carry on.

The level is INFO. Basket's __str__ prints "str called". The program runs log.debug(f"basket: {basket}") and then log.debug("basket: %s", basket). How many times is "str called" printed?

  • Never
  • Twice
  • Once
  • Three times

Answer: Once. The f-string is built before debug() is called. With %s, logging fills the message only for a line it is going to show, and DEBUG lines aren't shown.

Outside any except block, with the format "%(levelname)s %(message)s": log.error("charge failed", exc_info=True). What prints?

  • ERROR charge failed
  • It stops with ValueError
  • ERROR charge failed, then a traceback of the error() call
  • ERROR charge failed, then NoneType: None

Answer: ERROR charge failed, then NoneType: None. exc_info=True records the error being handled. There is none, so the traceback part says NoneType: None.

A program runs logging.debug("loading settings"), then logging.basicConfig(stream=sys.stdout, level=logging.DEBUG, format="%(message)s"), then logging.debug("settings loaded") and print("end"). What does standard output show?

  • end
  • settings loaded, then end
  • loading settings, settings loaded, end
  • It stops with ValueError

Answer: end. The first logging.debug() call set the root logger up (standard error, level WARNING), so the later basicConfig() did nothing and no DEBUG line is shown.

load() runs log.error("load failed") in its except block and re-raises; the caller catches the same error and runs log.error("request failed"). The format is "%(levelname)s %(message)s". What prints?

  • ERROR request failed
  • ERROR load failed, then ERROR request failed
  • ERROR load failed
  • It stops with ValueError

Answer: ERROR load failed, then ERROR request failed. Each except block logs it, so one failure gives two error lines. Log an error once, where it's handled.

Continue this course

Frequently asked questions

Is logging safe to use from several threads?

Yes: each handler takes a lock while it writes a line, so lines from different threads don't mix. Four threads writing 2,000 lines each through one handler gave 8,000 whole lines on Python 3.10–3.14. This site's editor can't start threads; Concurrency in Python: Threads vs Processes covers them.

How do I get warnings from the warnings module into my logs?

Call logging.captureWarnings(True). From then on, a warning goes to the logger named py.warnings at level WARNING instead of straight to standard error, so your handlers and formats apply to it. The warnings Module covers warnings themselves.

Can every line of one request carry its id without passing the id around?

Yes, with a contextvars.ContextVar that a handler filter reads, like the request-id filter above. Set the variable when a request starts; each asyncio task gets its own copy, so two requests handled at once keep their own ids. AsyncIO: Event Loop, Tasks & Futures covers tasks.

Can I keep my logging set-up in a file?

dictConfig() takes a dictionary, and a dictionary can come from a JSON file, as Working with JSON shows. Keep "disable_existing_loggers": False to leave existing loggers enabled; otherwise existing non-root loggers neither named by the configuration nor descended from a named logger are disabled. logging.config.fileConfig() reads the older INI-style files that Config Files with configparser covers.

Why does logger.warn() work?

It's an old spelling of warning(). It still logs on Python 3.10–3.14, and adds a DeprecationWarning on standard error: The 'warn' method is deprecated, use 'warning' instead.

What is %(taskName)s in a format?

The name of the asyncio task that made the line, from Python 3.12: inside a task named fetch-1 a line can read fetch-1: fetching, and outside any task the field is None. Python 3.10 and 3.11 have no such field, nor does this site's editor, which runs Python 3.11: there every line with it fails with a --- Logging error --- report.

Why do log lines show up in yellow, after everything printed, in this site's editor?

They're on standard error. The editor shows what the program printed first, then each line written to standard error, marked ⚠️. A terminal shows the two streams in the order they were written.

Related lessons