Testing with pytest

Published by Brayan K

Pytest is the standard testing framework for modern Python. Master professional testing strategies used by real engineering teams.

Part of the free Python course at LearnCodingFast — hands-on lessons with examples you run in your browser, plus practice exercises and a quick quiz.

assert: a check that stops the program when it's false

A test is a small function that calls your code with known input and checks the result. pytest finds such functions, runs them and reports which passed. This lesson writes tests, then gives them what they need: setup that runs before each one (fixtures), one test per case (parametrize), and stand-ins for what a test can't control (unittest.mock). Testing with unittest covers the test classes of the standard library's own unittest.

Each check in a test is an assert statement. It does nothing when its condition is true, and stops the program with an AssertionError when it's false:

def total(prices):
    return sum(prices)

assert total([2, 3]) == 5
print("first check passed")
assert total([]) == 1
print("this line never runs")

# ✅ Expected output:
# first check passed

# ✅ Expected error:
# AssertionError

After a comma comes a message for the error:

def total(prices):
    return sum(prices) + 1

result = total([2, 3])
assert result == 5, f"expected 5, got {result}"

# ✅ Expected error:
# AssertionError: expected 5, got 6

assert is a statement, not a function, so brackets around both parts make a tuple, and a tuple that isn't empty is true: assert(total([2, 3]) == 6, "total is wrong") never fails. Python 3.10–3.14 warn about it on standard error before the program runs, SyntaxWarning: assertion is always true, perhaps remove parentheses?, and then run it anyway. And as Logging, Debugging & Error Handling at Scale showed, python -O removes every assert from a program, so a check your program relies on needs an if and a raise. Tests are where assert belongs.

pytest runs the functions whose names start with test

pytest is a package, installed with pip as Modules and Packages showed; Virtual Environments & Dependency Management Best Practices puts it in a project's own environment. In a terminal, in your project's folder:

python -m pip install pytest
python -m pytest

The second command finds the test files in that folder and the folders below it, imports each one and calls every test function in it. A test passes when it finishes without an error, and a failing assert raises one. The command's exit code is 0 when every test passed and 1 when one failed, which is what a service that runs your tests on every push looks at.

This site's editor has pytest too: it loads pytest 7.2.2 when a program imports it. It has no terminal, so the examples here write their files with write_text(), as Modern File Paths with pathlib showed, and start pytest from Python with pytest.main(). pytest's own report has lines that change from run to run, such as how long the run took, so the examples hide it and print one line per test instead.

The lines after the test files do that, and they're the same in every example on this page. ShowEachTest is a plugin, an object whose methods pytest calls after it imports a test file and after each test; it prints PASSED, FAILED, ERROR or SKIPPED with the test's name, and the first line of pytest's explanation of a failure. contextlib.redirect_stdout() sends pytest's own report into an io.StringIO, the in-memory file from Reading CSV & Excel Files, and contextlib: contextmanager, suppress, ExitStack covers it. run_tests() ends by printing pytest's exit code.

from pathlib import Path

Path("cart.py").write_text("""
def total(prices):
    return sum(prices)
""", encoding="utf-8")
Path("test_cart.py").write_text("""
from cart import total

def test_total_of_two_prices():
    assert total([2, 3]) == 5

def test_empty_cart_costs_nothing():
    assert total([]) == 0

def test_total_of_three_prices():
    assert total([2, 3, 4]) == 10
""", encoding="utf-8")

import contextlib
import io
import sys

import pytest


class ShowEachTest:
    def __init__(self):
        self.screen = sys.stdout

    def pytest_collectreport(self, report):
        if report.failed:
            print("ERROR", report.nodeid, file=self.screen)
            print("   ", report.longreprtext.splitlines()[-1].strip(), file=self.screen)

    def pytest_runtest_logreport(self, report):
        if report.when == "call" or report.outcome != "passed":
            word = report.outcome.upper()
            if report.failed and report.when != "call":
                word = "ERROR"
            print(word, report.nodeid, file=self.screen)
            for line in report.longreprtext.splitlines():
                if line.startswith("E "):
                    print("   ", line[1:].strip(), file=self.screen)
                    break


def run_tests(target):
    show = ShowEachTest()
    with contextlib.redirect_stdout(io.StringIO()):
        code = pytest.main([target], plugins=[show])
    print("exit code:", int(code))


run_tests("test_cart.py")


# ✅ Expected output:
# PASSED test_cart.py::test_total_of_two_prices
# PASSED test_cart.py::test_empty_cart_costs_nothing
# FAILED test_cart.py::test_total_of_three_prices
#     assert 9 == 10
# exit code: 1

pytest imported test_cart.py and called its three test_ functions. The third failed, and the line under it is the first line of pytest's explanation: the two values its assert compared. The same tests run in a terminal with python -m pytest test_cart.py print the file's name followed by ..F, a dot for each pass and an F for each failure, then a report on each failure and a last line that counts them and says how long the run took.

pytest collects files named test_*.py or *_test.py, functions whose names start with test, and the test methods of classes whose names start with Test, as long as the class has no __init__ method:

from pathlib import Path

Path("test_names.py").write_text("""
def test_found():
    assert True

def testAlsoFound():
    assert True

def check_not_found():
    assert False

class TestCart:
    def test_in_a_class(self):
        assert True

class CartTests:
    def test_class_name_does_not_start_with_test(self):
        assert False
""", encoding="utf-8")
Path("cart_test.py").write_text("""
def test_file_ending_in_test():
    assert True
""", encoding="utf-8")
Path("tests_cart.py").write_text("""
def test_file_name_not_collected():
    assert False
""", encoding="utf-8")

import contextlib
import io
import sys

import pytest


class ShowEachTest:
    def __init__(self):
        self.screen = sys.stdout

    def pytest_collectreport(self, report):
        if report.failed:
            print("ERROR", report.nodeid, file=self.screen)
            print("   ", report.longreprtext.splitlines()[-1].strip(), file=self.screen)

    def pytest_runtest_logreport(self, report):
        if report.when == "call" or report.outcome != "passed":
            word = report.outcome.upper()
            if report.failed and report.when != "call":
                word = "ERROR"
            print(word, report.nodeid, file=self.screen)
            for line in report.longreprtext.splitlines():
                if line.startswith("E "):
                    print("   ", line[1:].strip(), file=self.screen)
                    break


def run_tests(target):
    show = ShowEachTest()
    with contextlib.redirect_stdout(io.StringIO()):
        code = pytest.main([target], plugins=[show])
    print("exit code:", int(code))


run_tests(".")


# ✅ Expected output:
# PASSED cart_test.py::test_file_ending_in_test
# PASSED test_names.py::test_found
# PASSED test_names.py::testAlsoFound
# PASSED test_names.py::TestCart::test_in_a_class
# exit code: 0

check_not_found, the class CartTests and the file tests_cart.py never ran, so their assert False lines failed nothing. A test pytest doesn't collect can't fail: a misnamed function is not collected, rather than reported as a SKIPPED test.

A test can pass without checking anything

Only an error fails a test. A comparison on a line of its own works out True or False and throws the answer away, and pytest ignores what a test returns:

from pathlib import Path

Path("cart.py").write_text("""
def total(prices):
    return sum(prices)
""", encoding="utf-8")
Path("test_cart.py").write_text("""
from cart import total

def test_compares_but_never_checks():
    total([2, 3]) == 6

def test_returns_instead_of_asserting():
    return total([2, 3]) == 6

def test_asserts():
    assert total([2, 3]) == 6
""", encoding="utf-8")

import contextlib
import io
import sys

import pytest


class ShowEachTest:
    def __init__(self):
        self.screen = sys.stdout

    def pytest_collectreport(self, report):
        if report.failed:
            print("ERROR", report.nodeid, file=self.screen)
            print("   ", report.longreprtext.splitlines()[-1].strip(), file=self.screen)

    def pytest_runtest_logreport(self, report):
        if report.when == "call" or report.outcome != "passed":
            word = report.outcome.upper()
            if report.failed and report.when != "call":
                word = "ERROR"
            print(word, report.nodeid, file=self.screen)
            for line in report.longreprtext.splitlines():
                if line.startswith("E "):
                    print("   ", line[1:].strip(), file=self.screen)
                    break


def run_tests(target):
    show = ShowEachTest()
    with contextlib.redirect_stdout(io.StringIO()):
        code = pytest.main([target], plugins=[show])
    print("exit code:", int(code))


run_tests("test_cart.py")


# ✅ Expected output:
# PASSED test_cart.py::test_compares_but_never_checks
# PASSED test_cart.py::test_returns_instead_of_asserting
# FAILED test_cart.py::test_asserts
#     assert 5 == 6
# exit code: 1

The first two tests passed, though total([2, 3]) is 5, not 6. The tests in Design Patterns in Python (Singleton, Factory, Strategy) returned True or False, which suits a program that prints them; under pytest they need assert. pytest's report adds a warning for the second test, PytestReturnNotNoneWarning, asking "Did you mean to use assert instead of return?", but the test still counts as passed.

A failing assert shows the values it compared

In a program, a failing assert says only AssertionError, as the first example showed. pytest rewrites the asserts in the test files it imports, so a failure there says what each side was:

from pathlib import Path

Path("cart.py").write_text("""
def total(prices):
    return sum(prices) + 1

def find_user(users, name):
    return users.get(name, "nobody")
""", encoding="utf-8")
Path("test_messages.py").write_text("""
from cart import total, find_user

def test_total():
    assert total([2, 3]) == 5

def test_lists():
    assert sorted([3, 1, 2]) == [1, 2, 4]

def test_dicts():
    assert {"apple": 50, "pear": 75} == {"apple": 50, "pear": 80}

def test_membership():
    assert "kiwi" in ["apple", "pear"]

def test_missing_user():
    assert find_user({"ada": 1}, "bob") is None

def test_with_a_message():
    result = total([2, 3])
    assert result == 5, f"expected 5, got {result}"
""", encoding="utf-8")

import contextlib
import io
import sys

import pytest


class ShowEachTest:
    def __init__(self):
        self.screen = sys.stdout

    def pytest_collectreport(self, report):
        if report.failed:
            print("ERROR", report.nodeid, file=self.screen)
            print("   ", report.longreprtext.splitlines()[-1].strip(), file=self.screen)

    def pytest_runtest_logreport(self, report):
        if report.when == "call" or report.outcome != "passed":
            word = report.outcome.upper()
            if report.failed and report.when != "call":
                word = "ERROR"
            print(word, report.nodeid, file=self.screen)
            for line in report.longreprtext.splitlines():
                if line.startswith("E "):
                    print("   ", line[1:].strip(), file=self.screen)
                    break


def run_tests(target):
    show = ShowEachTest()
    with contextlib.redirect_stdout(io.StringIO()):
        code = pytest.main([target], plugins=[show])
    print("exit code:", int(code))


run_tests("test_messages.py")


# ✅ Expected output:
# FAILED test_messages.py::test_total
#     assert 6 == 5
# FAILED test_messages.py::test_lists
#     assert [1, 2, 3] == [1, 2, 4]
# FAILED test_messages.py::test_dicts
#     AssertionError: assert {'apple': 50, 'pear': 75} == {'apple': 50, 'pear': 80}
# FAILED test_messages.py::test_membership
#     AssertionError: assert 'kiwi' in ['apple', 'pear']
# FAILED test_messages.py::test_missing_user
#     AssertionError: assert 'nobody' is None
# FAILED test_messages.py::test_with_a_message
#     AssertionError: expected 5, got 6
# exit code: 1

The runner prints each first line as pytest writes it, with the AssertionError: it puts in front of some. pytest's report goes on from there: under assert 6 == 5 it adds + where 6 = total([2, 3]), the call that gave 6, and for two lists or two dictionaries it names the items that differ.

Floats need care, as Operators and Expressions showed with 0.1 + 0.2. By default, pytest.approx() accepts a relative difference within one part in a million of the expected value, or an absolute difference within 1e-12. The absolute tolerance matters near zero; either tolerance can be configured:

import pytest

print(0.1 + 0.2 == 0.3)
print(0.1 + 0.2 == pytest.approx(0.3))
print(pytest.approx(0.3))


# ✅ Expected output:
# False
# True
# 0.3 ± 3.0e-07

pytest.raises: passing only when that error happens

Exception Handling showed a function refusing bad input with raise. A test of that refusal has to pass when the error happens and fail when it doesn't. pytest.raises() does both. It's a context manager, used in a with block like the ones in Context Managers & the with Statement in Depth:

import pytest

def parse_qty(text):
    qty = int(text)
    if qty < 1:
        raise ValueError(f"quantity must be at least 1, got {qty}")
    return qty

with pytest.raises(ValueError, match="at least 1") as info:
    parse_qty("0")
print(info.type.__name__)
print(info.value)


# ✅ Expected output:
# ValueError
# quantity must be at least 1, got 0

info.value is the error itself, like the e of except ValueError as e. If nothing in the block raises, the test fails: pytest 9 says Failed: DID NOT RAISE ValueError, and pytest 7.2.2, this site's, Failed: DID NOT RAISE <class 'ValueError'>. Another kind of error isn't caught at all:

from pathlib import Path

Path("test_qty.py").write_text("""
import pytest

def parse_qty(text):
    qty = int(text)
    if qty < 1:
        raise ValueError(f"quantity must be at least 1, got {qty}")
    return qty

def test_zero_is_rejected():
    with pytest.raises(ValueError, match="at least 1"):
        parse_qty("0")

def test_a_word_is_rejected():
    with pytest.raises(ValueError):
        parse_qty("ten")

def test_nothing_is_rejected():
    with pytest.raises(ValueError):
        parse_qty(None)
""", encoding="utf-8")

import contextlib
import io
import sys

import pytest


class ShowEachTest:
    def __init__(self):
        self.screen = sys.stdout

    def pytest_collectreport(self, report):
        if report.failed:
            print("ERROR", report.nodeid, file=self.screen)
            print("   ", report.longreprtext.splitlines()[-1].strip(), file=self.screen)

    def pytest_runtest_logreport(self, report):
        if report.when == "call" or report.outcome != "passed":
            word = report.outcome.upper()
            if report.failed and report.when != "call":
                word = "ERROR"
            print(word, report.nodeid, file=self.screen)
            for line in report.longreprtext.splitlines():
                if line.startswith("E "):
                    print("   ", line[1:].strip(), file=self.screen)
                    break


def run_tests(target):
    show = ShowEachTest()
    with contextlib.redirect_stdout(io.StringIO()):
        code = pytest.main([target], plugins=[show])
    print("exit code:", int(code))


run_tests("test_qty.py")


# ✅ Expected output:
# PASSED test_qty.py::test_zero_is_rejected
# PASSED test_qty.py::test_a_word_is_rejected
# FAILED test_qty.py::test_nothing_is_rejected
#     TypeError: int() argument must be a string, a bytes-like object or a real number, not 'NoneType'
# exit code: 1

int(None) raised a TypeError, which pytest.raises(ValueError) let through, so the test failed on it. A subclass of the expected error does count: pytest.raises(LookupError) passes for a KeyError.

match is a regular expression, searched for anywhere in the message, as re.search() does in Regular Expressions (re). So part of the message is enough, and characters such as ( and ) need re.escape():

import re
import pytest

def charge(amount):
    if amount <= 0:
        raise ValueError(f"amount must be > 0 (got {amount})")
    return amount

with pytest.raises(ValueError, match="must be > 0"):
    charge(-1)
print("a search: part of the message is enough")

with pytest.raises(ValueError, match=re.escape("(got -1)")):
    charge(-1)
print("re.escape makes ( and ) plain characters")


# ✅ Expected output:
# a search: part of the message is enough
# re.escape makes ( and ) plain characters

With an unescaped ( the pattern itself is broken, and the test fails on the regular-expression error, missing ), unterminated subpattern, before the message is checked at all. The raising call also ends the with block, so nothing after it inside the block runs:

import pytest

with pytest.raises(ValueError):
    int("ten")
    print("this line never runs")
print("after the with block")


# ✅ Expected output:
# after the with block

Keep only the call that should fail inside the block.

Fixtures: a fresh object for each test

Design Patterns in Python showed one test leaving tea in a shared cart for the next, and promised that pytest gives each test a fresh object. A fixture does that: a function marked with the @pytest.fixture decorator. A test asks for it by naming it as a parameter, and pytest calls the fixture and passes in what it returns:

from pathlib import Path

Path("test_cart.py").write_text("""
import pytest

class Cart:
    def __init__(self):
        self.items = []

@pytest.fixture
def cart():
    return Cart()

def test_add_tea(cart):
    cart.items.append("tea")
    assert cart.items == ["tea"]

def test_starts_empty(cart):
    assert cart.items == []
""", encoding="utf-8")

import contextlib
import io
import sys

import pytest


class ShowEachTest:
    def __init__(self):
        self.screen = sys.stdout

    def pytest_collectreport(self, report):
        if report.failed:
            print("ERROR", report.nodeid, file=self.screen)
            print("   ", report.longreprtext.splitlines()[-1].strip(), file=self.screen)

    def pytest_runtest_logreport(self, report):
        if report.when == "call" or report.outcome != "passed":
            word = report.outcome.upper()
            if report.failed and report.when != "call":
                word = "ERROR"
            print(word, report.nodeid, file=self.screen)
            for line in report.longreprtext.splitlines():
                if line.startswith("E "):
                    print("   ", line[1:].strip(), file=self.screen)
                    break


def run_tests(target):
    show = ShowEachTest()
    with contextlib.redirect_stdout(io.StringIO()):
        code = pytest.main([target], plugins=[show])
    print("exit code:", int(code))


run_tests("test_cart.py")


# ✅ Expected output:
# PASSED test_cart.py::test_add_tea
# PASSED test_cart.py::test_starts_empty
# exit code: 0

pytest called cart() once for each test, so the tea added in test_add_tea was gone by test_starts_empty. A fixture's scope says how long its result is kept. The default, "function", makes one per test; "class", "module", "package" and "session" share one object across more tests, which brings the leak back:

from pathlib import Path

Path("test_cart.py").write_text("""
import pytest

class Cart:
    def __init__(self):
        self.items = []

@pytest.fixture(scope="module")
def cart():
    return Cart()

def test_add_tea(cart):
    cart.items.append("tea")
    assert cart.items == ["tea"]

def test_starts_empty(cart):
    assert cart.items == []
""", encoding="utf-8")

import contextlib
import io
import sys

import pytest


class ShowEachTest:
    def __init__(self):
        self.screen = sys.stdout

    def pytest_collectreport(self, report):
        if report.failed:
            print("ERROR", report.nodeid, file=self.screen)
            print("   ", report.longreprtext.splitlines()[-1].strip(), file=self.screen)

    def pytest_runtest_logreport(self, report):
        if report.when == "call" or report.outcome != "passed":
            word = report.outcome.upper()
            if report.failed and report.when != "call":
                word = "ERROR"
            print(word, report.nodeid, file=self.screen)
            for line in report.longreprtext.splitlines():
                if line.startswith("E "):
                    print("   ", line[1:].strip(), file=self.screen)
                    break


def run_tests(target):
    show = ShowEachTest()
    with contextlib.redirect_stdout(io.StringIO()):
        code = pytest.main([target], plugins=[show])
    print("exit code:", int(code))


run_tests("test_cart.py")


# ✅ Expected output:
# PASSED test_cart.py::test_add_tea
# FAILED test_cart.py::test_starts_empty
#     AssertionError: assert ['tea'] == []
# exit code: 1

A shared fixture suits something slow to make that the tests only read, such as a connection to a test database. A test that names a fixture that doesn't exist never runs:

from pathlib import Path

Path("test_cart.py").write_text("""
import pytest

class Cart:
    def __init__(self):
        self.items = []

@pytest.fixture
def cart():
    return Cart()

def test_starts_empty(carts):
    assert carts.items == []
""", encoding="utf-8")

import contextlib
import io
import sys

import pytest


class ShowEachTest:
    def __init__(self):
        self.screen = sys.stdout

    def pytest_collectreport(self, report):
        if report.failed:
            print("ERROR", report.nodeid, file=self.screen)
            print("   ", report.longreprtext.splitlines()[-1].strip(), file=self.screen)

    def pytest_runtest_logreport(self, report):
        if report.when == "call" or report.outcome != "passed":
            word = report.outcome.upper()
            if report.failed and report.when != "call":
                word = "ERROR"
            print(word, report.nodeid, file=self.screen)
            for line in report.longreprtext.splitlines():
                if line.startswith("E "):
                    print("   ", line[1:].strip(), file=self.screen)
                    break


def run_tests(target):
    show = ShowEachTest()
    with contextlib.redirect_stdout(io.StringIO()):
        code = pytest.main([target], plugins=[show])
    print("exit code:", int(code))


run_tests("test_cart.py")


# ✅ Expected output:
# ERROR test_cart.py::test_starts_empty
#     fixture 'carts' not found
# exit code: 1

pytest reports ERROR, not FAILED: the problem is in the test's setup, before any check ran. Its report lists the fixtures that do exist, without a "Did you mean". Calling a fixture function yourself, as in cart() inside a test, doesn't work either: pytest stops it with an error saying fixtures are not meant to be called directly.

A yield fixture cleans up, even after a failure

A fixture that has to clean up after the test yields its value instead of returning it, and the code after yield runs once the test is over. In a @contextmanager function, as Context Managers & the with Statement in Depth showed, that code is skipped when the block fails, unless it's in a finally. A fixture is different: pytest runs it whether the test passed or failed. Here each step adds a line to a file, printed after the run, since pytest holds back what tests print:

from pathlib import Path

Path("test_order.py").write_text(r"""
import pytest

def note(text):
    with open("events.txt", "a", encoding="utf-8") as log:
        log.write(text + "\n")

@pytest.fixture
def database():
    note("open database")
    yield {"users": []}
    note("close database")

@pytest.fixture
def user(database):
    note("add user ada")
    database["users"].append("ada")
    yield "ada"
    note("remove user ada")
    database["users"].remove("ada")

def test_user_is_stored(user, database):
    note("test runs")
    assert database["users"] == []
""", encoding="utf-8")

import contextlib
import io
import sys

import pytest


class ShowEachTest:
    def __init__(self):
        self.screen = sys.stdout

    def pytest_collectreport(self, report):
        if report.failed:
            print("ERROR", report.nodeid, file=self.screen)
            print("   ", report.longreprtext.splitlines()[-1].strip(), file=self.screen)

    def pytest_runtest_logreport(self, report):
        if report.when == "call" or report.outcome != "passed":
            word = report.outcome.upper()
            if report.failed and report.when != "call":
                word = "ERROR"
            print(word, report.nodeid, file=self.screen)
            for line in report.longreprtext.splitlines():
                if line.startswith("E "):
                    print("   ", line[1:].strip(), file=self.screen)
                    break


def run_tests(target):
    show = ShowEachTest()
    with contextlib.redirect_stdout(io.StringIO()):
        code = pytest.main([target], plugins=[show])
    print("exit code:", int(code))


run_tests("test_order.py")

print(Path("events.txt").read_text(encoding="utf-8"), end="")


# ✅ Expected output:
# FAILED test_order.py::test_user_is_stored
#     AssertionError: assert ['ada'] == []
# exit code: 1
# open database
# add user ada
# test runs
# remove user ada
# close database

pytest set up database first, because user asks for it, and took them down in the opposite order. Both clean-ups ran, though the test failed.

conftest.py, and the fixtures pytest provides

A fixture in a file named conftest.py can be used by every test file in that folder and the folders below it, without an import:

from pathlib import Path

Path("conftest.py").write_text("""
import pytest

@pytest.fixture
def prices():
    return {"apple": 50, "pear": 75}
""", encoding="utf-8")
Path("test_apple.py").write_text("""
def test_apple(prices):
    assert prices["apple"] == 50
""", encoding="utf-8")
Path("test_pear.py").write_text("""
def test_pear(prices):
    assert prices["pear"] == 70
""", encoding="utf-8")

import contextlib
import io
import sys

import pytest


class ShowEachTest:
    def __init__(self):
        self.screen = sys.stdout

    def pytest_collectreport(self, report):
        if report.failed:
            print("ERROR", report.nodeid, file=self.screen)
            print("   ", report.longreprtext.splitlines()[-1].strip(), file=self.screen)

    def pytest_runtest_logreport(self, report):
        if report.when == "call" or report.outcome != "passed":
            word = report.outcome.upper()
            if report.failed and report.when != "call":
                word = "ERROR"
            print(word, report.nodeid, file=self.screen)
            for line in report.longreprtext.splitlines():
                if line.startswith("E "):
                    print("   ", line[1:].strip(), file=self.screen)
                    break


def run_tests(target):
    show = ShowEachTest()
    with contextlib.redirect_stdout(io.StringIO()):
        code = pytest.main([target], plugins=[show])
    print("exit code:", int(code))


run_tests(".")


# ✅ Expected output:
# PASSED test_apple.py::test_apple
# FAILED test_pear.py::test_pear
#     assert 75 == 70
# exit code: 1

pytest has fixtures of its own as well. tmp_path is a new, empty folder for each test, as a pathlib Path, and capsys holds what the code under test printed:

The folder is in a different place on each run, so a test checks what's in it, not where it is. Temporary Files & Directories covers the tempfile module, which makes such folders outside tests. A third fixture, monkeypatch, comes with patching below.

parametrize: one test per case

A loop inside one test stops at the first bad case. @pytest.mark.parametrize makes one test per case instead. It's a decorator that takes arguments, like @repeat(3) in Decorators & Advanced Features: a string naming the test's parameters, and a list with one tuple of values per case. The second row here is wrong, since half of 80 is 40:

The loop test stopped at the second row and never checked the third. The parametrized test ran each row on its own, so the report shows which case failed and that the others passed. pytest names each case after its values, joined by -. Strings appear as they are, pytest.param(..., id=...) names a case yourself, and values such as lists get the parameter's name and their position:

Two parametrize decorators give every combination. The one nearest the def is applied first, as Decorators & Advanced Features showed for stacked decorators, so its value comes first in each name:

unittest.mock: stand-ins for what a test can't control

A function that calls a web service, reads the clock or sends an email is hard to test as it is. The standard library's unittest.mock makes stand-ins. A Mock accepts any attribute and any call, returns what you tell it to, and records how it was called:

from unittest.mock import Mock

def user_name(api):
    return api.get("/user")["name"]

api = Mock()
api.get.return_value = {"id": 1, "name": "Ada"}
print(user_name(api))
api.get.assert_called_once_with("/user")
print(api.get.call_count, api.get.call_args)


# ✅ Expected output:
# Ada
# 1 call('/user')

assert_called_once_with() raises an AssertionError when the calls were different, which fails a test:

from unittest.mock import Mock

api = Mock()
api.get("/user")
api.get("/user")
try:
    api.get.assert_called_once_with("/user")
except AssertionError as err:
    print(err)

api = Mock()
api.get("/users")
try:
    api.get.assert_called_once_with("/user")
except AssertionError as err:
    for line in str(err).splitlines():
        print(line.strip())


# ✅ Expected output:
# Expected 'get' to be called once. Called 2 times.
# Calls: [call('/user'), call('/user')].
# expected call not found.
# Expected: get('/user')
# Actual: get('/users')

The second message is printed a line at a time with .strip() because newer Python releases indent its last line and older ones, this site's editor's among them, don't. Spell these checks in full. api.get.called_once_with("/user"), without assert_, stops with an AttributeError on Python 3.12 and later, but 3.10, 3.11 and this site's editor treat it as a call to one more attribute of the mock, which checks nothing.

side_effect makes a call raise an error, or return the next value from a list each time:

Once the list runs out, the next call raises StopIteration. A Mock doesn't support Python's special methods, such as the __len__ that len() calls; Magic Methods & the Python Data Model covers those. A MagicMock supports them:

Don't print a mock itself in a check: its text includes an id that changes from run to run.

Patch the name where it's looked up

unittest.mock.patch() swaps a name for a Mock while a with block runs, then puts the original back. Choosing which name to patch is where it usually goes wrong. shop.py below does from rates import current_rate, which copies the name into shop's own namespace, so shop never looks in rates for it again:

Patching rates.current_rate changed the name in rates and left shop's copy alone, so the price stayed 12.0. Patch the module that uses the name, here shop. Had shop done import rates and called rates.current_rate(), it would look the name up in rates on every call, and patching rates.current_rate would work.

Replacing a name by hand is easy to get wrong: if a check fails before the line that puts it back, the replacement stays. patch puts the original back however the block ends:

In this site's editor every Run on a page shares one Python, so a hand-made replacement of a standard-library function, such as time.time, lasts until the page is reloaded. patch also works as a decorator: above a function, it hands the Mock in as an argument and puts the original back when the function returns. Inside a pytest test, the monkeypatch fixture does the same job, and pytest undoes its changes when the test is over:

The real rate was back for the second test, and after the run.

A Mock accepts anything, unless it has a spec

A plain Mock accepts a misspelled method as readily as a real one, so a test can pass while the code calls something that doesn't exist. Mock(spec=Api) accepts only the attributes Api has:

create_autospec() goes further and checks each call's arguments against the real function's parameters:

Where tests live, and how they import your code

Module & Package Architecture for Large Codebases laid out a project with its package beside a tests folder, and left the tests to this lesson. Shortened to the parts that matter here:

tests/test_prices.py starts with from shop.prices import with_vat. Run from shop-project/, the two ways of starting pytest behave differently. Measured on Python 3.10–3.14 with pytest 9.1.1:

Any one of these makes pytest work as well: an empty conftest.py in shop-project/, an empty tests/__init__.py, or these lines in pyproject.toml:

With the package inside a src folder, that line becomes pythonpath = ["src"], or the package is installed, which Packaging & Publishing Python Libraries to PyPI covers. Two test files with the same name in different folders, such as tests/unit/test_utils.py and tests/integration/test_utils.py, stop pytest with import file mismatch, unless tests and both folders have an __init__.py.

In this site's editor, pytest.main() runs inside your program, whose folder is already on sys.path, so it behaves like python -m pytest:

A few options save time in a terminal:

-q prints less. A file name, :: and a test's name run that one test. -k runs the tests whose names contain a word, -x stops at the first failure, and --co lists the tests without running them. The exit code is 0 when every test passed, 1 when a test failed, 2 when pytest was interrupted, for example by a test file it couldn't import, 4 for a mistake in the command, such as a file that isn't there, and 5 when it found no tests.

Skipping a test

@pytest.mark.skip skips a test, and @pytest.mark.skipif skips it when a condition is true. Both take a reason:

@pytest.mark.xfail(reason="known bug") marks a test that's expected to fail: pytest runs it and counts it as xfailed rather than failed. With -rsx, pytest's report lists the skipped and xfailed tests with their reasons. A mark of your own, such as @pytest.mark.slow, needs registering under markers in pyproject.toml, or pytest warns Unknown pytest.mark.slow - is this a typo?. Once it's registered, python -m pytest -m "not slow" leaves those tests out.

Practice: parametrize a discount test

Fill in the decorator's name and the missing parameter, so the three rows run as three tests. While a blank is left, the Run prints an ERROR line for the file instead of a test: pytest couldn't load it, and the line under it says why.

On your own: test initials()

names.py has a function that turns a name into initials: "ada lovelace" gives "A.L.", and a name that's only spaces raises a ValueError, name is empty. Write test_names.py:

As given, the program prints exit code: 5: pytest found no tests yet.

Then change part[0].upper() to part[0] in names.py and run the solution again. The three initials tests fail, the first with AssertionError: assert 'a.l.' == 'A.L.', and the blank-name test still passes: the tests catch the bug.

Practice quiz

total([2, 3]) returns 5. What does pytest report for def test_total(): total([2, 3]) == 6 ?

  • FAILED, with assert 5 == 6
  • PASSED
  • ERROR
  • SKIPPED

Answer: PASSED. The comparison gives False, and nothing uses it. A test fails only when something in it raises an error, and it takes an assert to turn False into one.

total([2, 3]) returns 5. What happens on the line assert(total([2, 3]) == 6, "total is wrong")?

  • AssertionError: total is wrong
  • TypeError: assert takes one argument
  • Nothing stops: the check passes and the next line runs
  • SyntaxError

Answer: Nothing stops: the check passes and the next line runs. The brackets make a tuple of two items, and a tuple that isn't empty is true. Python warns that the assertion is always true and runs on.

A fixture cart has scope="module". test_add_tea appends "tea"; test_starts_empty asserts cart.items == []. What does pytest report?

  • test_starts_empty fails with assert ['tea'] == []
  • Both pass
  • test_add_tea fails
  • Both fail

Answer: test_starts_empty fails with assert ['tea'] == []. scope="module" makes one Cart for the whole file, so the tea the first test added is still there in the second.

CASES has three rows, and the second is wrong. One test loops over CASES; another is parametrized with CASES. What fails?

  • The loop test 3 times and the parametrized test once
  • The loop test once and all 3 parametrized tests
  • The loop test once, and 1 of the 3 parametrized tests
  • Nothing: a loop and parametrize both check every row

Answer: The loop test once, and 1 of the 3 parametrized tests. The loop test stops at the second row and never checks the third. parametrize makes three tests, and only the one for the wrong row fails.

shop.py does from rates import current_rate, and current_rate() returns 0.2. What does with patch("rates.current_rate", return_value=0.5): print(shop.price_with_tax(10)) print?

  • 15.0
  • 12.0
  • AttributeError
  • 10

Answer: 12.0. The import copied the name into shop. Patching rates changes the name in rates, not shop's copy, so shop still uses the real rate.

A yield fixture has no try/finally, and the test that uses it fails. Does the code after its yield run?

  • No, as with a @contextmanager function without finally
  • Only for scope="module"
  • Only when the test passes
  • Yes: pytest runs it after the test either way

Answer: Yes: pytest runs it after the test either way. pytest runs the code after a fixture's yield once the test is over, whether the test passed or failed.

int(None) raises TypeError. What does pytest report for a test whose body is with pytest.raises(ValueError): parse_qty(None) ?

  • FAILED, with the TypeError
  • PASSED: an error happened
  • SKIPPED
  • ERROR in setup

Answer: FAILED, with the TypeError. pytest.raises(ValueError) catches only ValueError and its subclasses. The TypeError goes on, and the test fails on it.

The fixture is named cart, and the test is def test_starts_empty(carts): ... What does pytest report?

  • FAILED with NameError: name 'carts' is not defined
  • PASSED
  • SKIPPED
  • ERROR, with fixture 'carts' not found

Answer: ERROR, with fixture 'carts' not found. pytest looks for a fixture with the parameter's name and finds none, so it reports an ERROR: the test never ran.

Continue this course

Frequently asked questions

Should I use pytest or unittest?

unittest is in the standard library and writes tests as methods of a TestCase class, with checks such as self.assertEqual(); Testing with unittest covers it. pytest runs those TestCase classes too, so a project can move to plain test functions one file at a time.

How do I test async def functions?

pytest needs a plugin for them. On its own, pytest 9 fails an async def test with "async def functions are not natively supported." and names plugins to install, such as anyio and pytest-asyncio. pytest 7.2.2, the version this site's editor loads, skips such a test with a warning instead. Advanced Async & Await Patterns covers async code.

What is the mocker parameter I see in other people's tests?

A fixture from the pytest-mock plugin, which isn't part of pytest. Neither this site's editor nor the build that checks these examples has it. unittest.mock's patch() and pytest's monkeypatch, shown in this lesson, do the same jobs.

Why do .pytest_cache and __pycache__ folders appear?

pytest keeps notes on the last run in .pytest_cache, such as which tests failed: python -m pytest --lf runs only those again. __pycache__ holds the test files pytest rewrote for its assert messages. Both are made again when they're missing, so they can be deleted, and they don't belong in version control.

How do I see what my tests print?

pytest holds it back. A failing test's prints appear in the report, under Captured stdout call, and a passing test's don't appear at all. python -m pytest -s lets every print through as it happens.

Do tests run in the order they're written?

Yes: files in order of their names, and the tests in each file from top to bottom, as the examples here show. Don't rely on it, though. A test that needs another to run first fails when it's run on its own, with -k or by its name.

Why does pytest's report look different on GitHub Actions?

GitHub Actions sets the environment variable CI to true, and pytest checks for it. A failing comparison of two lists then shows the whole difference instead of "Use -v to get more diff", and the summary lines at the end aren't shortened. The same tests pass and fail; only the report changes.

Related lessons