Metaprogramming & Introspection

Reviewed & published by Brayan K

Master Python's introspection tools, inspect module, dynamic code generation, and metaclasses

Part of the free Python course at LearnCodingFast — hands-on lessons with examples you run in your browser, plus practice exercises and a quick quiz.

What Is Metaprogramming?

Metaprogramming means writing code that treats functions, classes, and modules as data and manipulates them.

• Automatically registering classes
• Inspecting function signatures
• Modifying functions with decorators
• Generating methods dynamically
• Building plugin systems
• Auto-validating data

Python makes this easy because functions, classes, and modules are all objects that can be inspected and modified.

Introspection Basics: dir(), type(), vars()

Python provides built-in functions for basic introspection.

# Basic introspection tools

class User:
    def __init__(self, name, age):
        self.name = name
        self.age = age
    
    def greet(self):
        return f"Hello, I'm {self.name}"

user = User("Alice", 30)

# dir() - shows all attributes and methods
print("Attributes and methods:")
print([attr for attr in dir(user) if not attr.startswith('_')])

# type() - returns the class
print(f"\nType: {type(user)}")
print(f"Class name: {type(user).__name__}")

# vars() - returns object's __dict__ (instance attributes)
print(f"\nInstance attributes: {vars(user)}")

# hasattr, getattr, setattr
print(f"\nHas 'name'? {hasattr(user, 'name')}")
print(f"Get 'name': {getattr(user, 'name')}")

setattr(user, 'email', '[email protected]')
print(f"After setattr: {vars(user)}")

# ✅ Expected output:
# Attributes and methods:
# ['age', 'greet', 'name']
#
# Type: <class '__main__.User'>
# Class name: User
#
# Instance attributes: {'name': 'Alice', 'age': 30}
#
# Has 'name'? True
# Get 'name': Alice
# After setattr: {'name': 'Alice', 'age': 30, 'email': '[email protected]'}

Worked Example: Introspecting One Object End to End

Those built-ins are easier to hold in your head when you watch them all applied to the same object. Read the comments — each one states what that line prints and why — then run it and check the output block at the bottom.

class Product:
    """A thing in a shop."""
    category = "general"          # class attribute: shared by every Product

    def __init__(self, name, price):
        self.name = name          # instance attributes: unique to each object
        self.price = price

    def label(self):
        return f"{self.name}: £{self.price:.2f}"

p = Product("Kettle", 24.5)

# type() hands back the class itself; __name__ is its plain name as text.
print("class      :", type(p).__name__)        # Product

# vars() shows ONLY the instance attributes, as a dictionary.
print("instance   :", vars(p))                 # {'name': 'Kettle', 'price': 24.5}

# hasattr asks "does this name exist?" without risking an AttributeError.
print("has price? :", hasattr(p, "price"))     # True
print("has stock? :", hasattr(p, "stock"))     # False

# getattr fetches an attribute BY NAME, where the name is a string your
# program can decide while it runs. This is the heart of metaprogramming.
field = "name"
print("getattr    :", getattr(p, field))       # Kettle

# A third argument is a fallback, returned instead of raising AttributeError.
print("fallback   :", getattr(p, "stock", 0))  # 0

# setattr creates (or overwrites) an attribute by name.
setattr(p, "stock", 7)
print("after set  :", vars(p))

# Methods are attributes too: getattr gives you the bound method, then you call it.
method = getattr(p, "label")
print("called     :", method())                # Kettle: £24.50

# dir() lists every name on the object. Filter out the dunder ("double
# underscore") names Python adds, and keep only the callable ones, and what
# is left is the object's own public methods.
print("methods    :", [n for n in dir(p) if callable(getattr(p, n)) and not n.startswith("_")])

# ✅ Expected output:
# class      : Product
# instance   : {'name': 'Kettle', 'price': 24.5}
# has price? : True
# has stock? : False
# getattr    : Kettle
# fallback   : 0
# after set  : {'name': 'Kettle', 'price': 24.5, 'stock': 7}
# called     : Kettle: £24.50
# methods    : ['label']

The inspect Module

The inspect module provides powerful tools for analyzing Python objects deeply.

import inspect

def greet(name: str, greeting: str = "Hello") -> str:
    """Greet someone with a custom message."""
    return f"{greeting}, {name}!"

# Get function signature
sig = inspect.signature(greet)
print("Signature:", sig)

# Get parameters
print("\nParameters:")
for param_name, param in sig.parameters.items():
    print(f"  {param_name}:")
    print(f"    default: {param.default}")
    print(f"    annotation: {param.annotation}")

# Check return annotation
print(f"\nReturn type: {sig.return_annotation}")

# Get source code
print("\nSource code:")
print(inspect.getsource(greet))

# Get docstring
print(f"Docstring: {inspect.getdoc(greet)}")

# Check what it is
print(f"\nIs function? {inspect.isfunction(greet)}")
print(f"Is coroutine? {inspect.iscoroutinefunction(greet)}")

# ✅ Expected output:
# Signature: (name: str, greeting: str = 'Hello') -> str
#
# Parameters:
#   name:
#     default: <class 'inspect._empty'>
#     annotation: <class 'str'>
#   greeting:
#     default: Hello
#     annotation: <class 'str'>
#
# Return type: <class 'str'>
#
# Source code:
# def greet(name: str, greeting: str = "Hello") -> str:
#     """Greet someone with a custom message."""
#     return f"{greeting}, {name}!"
#
# Docstring: Greet someone with a custom message.
#
# Is function? True
# Is coroutine? False

Inspecting Function Signatures

Extract detailed information about function parameters, defaults, and type hints.

import inspect

def complex_function(
    required: str,
    optional: int = 10,
    *args: str,
    keyword_only: bool,
    with_default: float = 3.14,
    **kwargs: dict
) -> list:
    """Complex function with various parameter types."""
    pass

sig = inspect.signature(complex_function)

print("Detailed parameter analysis:\n")

for name, param in sig.parameters.items():
    print(f"Parameter: {name}")
    print(f"  Kind: {param.kind}")
    print(f"  Default: {param.default}")
    print(f"  Annotation: {param.annotation}")
    print()

# Parameter kinds
print("Parameter kinds explained:")
print("  POSITIONAL_ONLY: Can only be passed positionally")
print("  POSITIONAL_OR_KEYWORD: Normal parameters")
print("  VAR_POSITIONAL: *args")
print("  KEYWORD_ONLY: Must use keyword")
print("  VAR_KEYWORD: **kwargs")

# This is how FastAPI auto-validates API inputs!

# ✅ Expected output:
# Detailed parameter analysis:
#
# Parameter: required
#   Kind: POSITIONAL_OR_KEYWORD
#   Default: <class 'inspect._empty'>
#   Annotation: <class 'str'>
#
# Parameter: optional
#   Kind: POSITIONAL_OR_KEYWORD
#   Default: 10
#   Annotation: <class 'int'>
#
# Parameter: args
#   Kind: VAR_POSITIONAL
#   Default: <class 'inspect._empty'>
#   Annotation: <class 'str'>
#
# Parameter: keyword_only
#   Kind: KEYWORD_ONLY
#   Default: <class 'inspect._empty'>
#   Annotation: <class 'bool'>
#
# Parameter: with_default
#   Kind: KEYWORD_ONLY
#   Default: 3.14
#   Annotation: <class 'float'>
#
# Parameter: kwargs
#   Kind: VAR_KEYWORD
#   Default: <class 'inspect._empty'>
#   Annotation: <class 'dict'>
#
# Parameter kinds explained:
#   POSITIONAL_ONLY: Can only be passed positionally
#   POSITIONAL_OR_KEYWORD: Normal parameters
#   VAR_POSITIONAL: *args
#   KEYWORD_ONLY: Must use keyword
#   VAR_KEYWORD: **kwargs

Detecting Types of Objects

Inspect can identify exactly what kind of object you're dealing with.

import inspect

def regular_function():
    pass

async def async_function():
    pass

def generator_function():
    yield 1

class MyClass:
    def method(self):
        pass

obj = MyClass()

# Type detection
checks = [
    ("regular_function", regular_function),
    ("async_function", async_function),
    ("generator_function", generator_function),
    ("MyClass", MyClass),
    ("obj", obj),
]

print("Object type detection:\n")

for name, item in checks:
    print(f"{name}:")
    print(f"  is function: {inspect.isfunction(item)}")
    print(f"  is method: {inspect.ismethod(item)}")
    print(f"  is class: {inspect.isclass(item)}")
    print(f"  is coroutine function: {inspect.iscoroutinefunction(item)}")
    print(f"  is generator function: {inspect.isgeneratorfunction(item)}")
    print()

# Used for: plugin loading, auto-registration, CLI builders

# ✅ Expected output:
# Object type detection:
#
# regular_function:
#   is function: True
#   is method: False
#   is class: False
#   is coroutine function: False
#   is generator function: False
#
# async_function:
#   is function: True
#   is method: False
#   is class: False
#   is coroutine function: True
#   is generator function: False
#
# generator_function:
#   is function: True
#   is method: False
#   is class: False
#   is coroutine function: False
#   is generator function: True
#
# MyClass:
#   is function: False
#   is method: False
#   is class: True
#   is coroutine function: False
#   is generator function: False
#
# obj:
#   is function: False
#   is method: False
#   is class: False
#   is coroutine function: False
#   is generator function: False

🎯 Your Turn: Read and Write Attributes by Name

Here is the pattern almost every configuration loader uses: a list of attribute names decided at runtime, checked and read one by one. Three blanks, one function each. Fill them in and match the expected output.

# 🎯 YOUR TURN — replace each ___ with the right built-in function.

class Settings:
    def __init__(self):
        self.theme = "dark"
        self.volume = 7

s = Settings()
wanted = ["theme", "volume", "language"]   # names decided at runtime, as strings

for name in wanted:
    if ___(s, name):                       # 👉 which one asks "does this attribute exist?"
        print(name, "=", ___(s, name))     # 👉 which one FETCHES it by name?
    else:
        print(name, "= <missing>")

___(s, "language", "en")                   # 👉 which one CREATES an attribute by name?
print("after:", vars(s))

# ✅ Expected output:
# theme = dark
# volume = 7
# language = <missing>
# after: {'theme': 'dark', 'volume': 7, 'language': 'en'}

Getting the Call Stack

Inspect where your code is being called from - essential for debugging and logging.

import inspect

def show_caller_info():
    """Show information about who called this function."""
    # stack[0] is this function
    # stack[1] is the caller
    frame_info = inspect.stack()[1]
    
    print(f"Called from function: {frame_info.function}")
    print(f"  File: {frame_info.filename}")
    print(f"  Line: {frame_info.lineno}")
    print(f"  Code: {frame_info.code_context[0].strip()}")

def level_three():
    print("Level 3:")
    show_caller_info()

def level_two():
    print("\nLevel 2:")
    show_caller_info()
    level_three()

def level_one():
    print("Level 1:")
    show_caller_info()
    level_two()

# Run the nested calls
level_one()

print("\n--- Full Stack Trace ---")

def show_full_stack():
    stack = inspect.stack()
    print(f"Stack has {len(stack)} frames:\n")
    for i, frame in enumerate(stack):
        print(f"{i}. {frame.function} at line {frame.lineno}")

def deep_call():
    show_full_stack()

deep_call()

Dynamic Function Generation

Create functions at runtime based on configuration or input.

import inspect

# Factory pattern - create functions dynamically
def create_validator(field_name, field_type):
    """Generate a validation function for a field."""
    def validator(value):
        if not isinstance(value, field_type):
            raise TypeError(
                f"{field_name} must be {field_type.__name__}, "
                f"got {type(value).__name__}"
            )
        return value
    
    # Set a helpful name
    validator.__name__ = f"validate_{field_name}"
    return validator

# Create validators
validate_age = create_validator("age", int)
validate_name = create_validator("name", str)

print("Testing validators:")
try:
    print(validate_age(25))  # OK
    print(validate_name("Alice"))  # OK
    print(validate_age("not a number"))  # Error
except TypeError as e:
    print(f"Error: {e}")

# Inspect the generated function
print(f"\nGenerated function name: {validate_age.__name__}")

# Create operation functions
def create_operation(op_name, op_func):
    """Create a named operation function."""
    def operation(a, b):
        result = op_func(a, b)
        print(f"{op_name}({a}, {b}) = {result}")
        return result
    
    operation.__name__ = op_name
    return operation

add = create_operation("add", lambda a, b: a + b)
multiply = create_operation("multiply", lambda a, b: a * b)

print("\nDynamic operations:")
add(5, 3)
multiply(4, 7)

# ✅ Expected output:
# Testing validators:
# 25
# Alice
# Error: age must be int, got str
#
# Generated function name: validate_age
#
# Dynamic operations:
# add(5, 3) = 8
# multiply(4, 7) = 28

Intercepting Attribute Access

Use __getattr__, __setattr__ to intercept and control attribute access.

# Lazy loading with __getattr__
class LazyConfig:
    """Config that generates values on first access."""
    
    def __getattr__(self, name):
        if name.startswith('db_'):
            # Simulate loading from database
            value = f"<loaded {name} from DB>"
            # Cache it
            setattr(self, name, value)
            return value
        raise AttributeError(f"'{name}' not found")

config = LazyConfig()
print("First access (loads from DB):", config.db_host)
print("Second access (cached):", config.db_host)

# Tracking changes with __setattr__
class TrackedObject:
    """Object that logs all attribute changes."""
    
    def __init__(self):
        # Use object.__setattr__ to avoid recursion
        object.__setattr__(self, '_changes', [])
    
    def __setattr__(self, name, value):
        if name != '_changes':
            self._changes.append((name, value))
            print(f"Set {name} = {value}")
        object.__setattr__(self, name, value)
    
    def show_changes(self):
        print("\nChange history:")
        for name, value in self._changes:
            print(f"  {name} = {value}")

obj = TrackedObject()
obj.name = "Alice"
obj.age = 30
obj.email = "[email protected]"
obj.show_changes()

# This is how ORMs track changes for database updates!

# ✅ Expected output:
# First access (loads from DB): <loaded db_host from DB>
# Second access (cached): <loaded db_host from DB>
# Set name = Alice
# Set age = 30
# Set email = [email protected]
#
# Change history:
#   name = Alice
#   age = 30
#   email = [email protected]

Function Metadata & Annotations

Functions carry metadata that frameworks use for validation and documentation.

def api_endpoint(
    user_id: int,
    include_posts: bool = False,
    page: int = 1
) -> dict:
    """
    Fetch user data from the API.
    
    Args:
        user_id: The ID of the user
        include_posts: Whether to include user's posts
        page: Page number for pagination
    
    Returns:
        User data dictionary
    """
    return {"id": user_id, "posts": include_posts, "page": page}

# Access function metadata
print("Function metadata:\n")
print(f"Name: {api_endpoint.__name__}")
print(f"Docstring: {api_endpoint.__doc__[:50]}...")
print(f"\nAnnotations: {api_endpoint.__annotations__}")
print(f"Defaults: {api_endpoint.__defaults__}")
print(f"Module: {api_endpoint.__module__}")

# This is what FastAPI uses to generate:
# - API documentation
# - Request validation
# - Response schemas
# - OpenAPI specs

# Auto-generate API documentation
def document_api(func):
    """Generate API documentation from function."""
    print(f"\n=== API Endpoint: {func.__name__} ===")
    print(f"Description: {func.__doc__.split('Args:')[0].strip()}")
    print("\nParameters:")
    
    annotations = func.__annotations__
    defaults = func.__defaults__ or ()
    
    for param, annotation in annotations.items():
        if param == 'return':
            continue
        is_required = param not in str(defaults)
        print(f"  - {param}: {annotation.__name__} " + 
              ("(required)" if is_required else "(optional)"))

document_api(api_endpoint)

# ✅ Expected output:
# Function metadata:
#
# Name: api_endpoint
# Docstring: 
#     Fetch user data from the API.
#     
#     Args:
# ...
#
# Annotations: {'user_id': <class 'int'>, 'include_posts': <class 'bool'>, 'page': <class 'int'>, 'return': <class 'dict'>}
# Defaults: (False, 1)
# Module: __main__
#
# === API Endpoint: api_endpoint ===
# Description: Fetch user data from the API.
#
# Parameters:
#   - user_id: int (required)
#   - include_posts: bool (required)
#   - page: int (required)

Inspecting Closures

Examine captured variables in closures - how decorators store state.

import inspect
import time

def create_counter(start=0, step=1):
    """Create a counter function with captured state."""
    count = start
    
    def counter():
        nonlocal count
        count += step
        return count
    
    return counter

# Create counters with different captured values
counter1 = create_counter(0, 1)
counter2 = create_counter(100, 10)

print("Counting:")
print(counter1())  # 1
print(counter1())  # 2
print(counter2())  # 110
print(counter2())  # 120

# Inspect the closure
print("\nInspecting counter1 closure:")
closure = counter1.__closure__
if closure:
    for i, cell in enumerate(closure):
        print(f"  Captured variable {i}: {cell.cell_contents}")

# Practical example: rate limiter
def rate_limiter(max_calls, period):
    """Create a rate limiting decorator."""
    calls = []
    
    def decorator(func):
        def wrapper(*args, **kwargs):
            now = time.time()
            # Remove old calls
            calls[:] = [t for t in calls if now - t < period]
            
            if len(calls) >= max_calls:
                raise Exception("Rate limit exceeded")
            
            calls.append(now)
            return func(*args, **kwargs)
        
        # Inspect the closure
        print(f"Rate limiter closure variables:")
        if wrapper.__closure__:
            for cell in wrapper.__closure__:
                print(f"  {type(cell.cell_contents).__name__}")
        
        return wrapper
    return decorator

@rate_limiter(max_calls=3, period=10)
def api_call():
    return "Success"

print("\nTesting rate limiter:")
for i in range(4):
    try:
        print(f"Call {i+1}: {api_call()}")
    except Exception as e:
        print(f"Call {i+1}: {e}")

Building a Plugin System

Use introspection to automatically discover and register plugins.

import inspect

# Base class for plugins
class Plugin:
    """Base plugin class."""
    name = "base"
    
    def execute(self):
        raise NotImplementedError

# Plugin implementations
class EmailPlugin(Plugin):
    name = "email"
    
    def execute(self):
        return "Sending email..."

class SMSPlugin(Plugin):
    name = "sms"
    
    def execute(self):
        return "Sending SMS..."

class SlackPlugin(Plugin):
    name = "slack"
    
    def execute(self):
        return "Posting to Slack..."

# Auto-discover plugins
def discover_plugins(base_class):
    """Find all subclasses of base_class."""
    plugins = {}
    
    # Get all classes in current module
    current_module = inspect.getmodule(base_class)
    
    for name, obj in inspect.getmembers(current_module):
        if (inspect.isclass(obj) and 
            issubclass(obj, base_class) and 
            obj is not base_class):
            
            # Register by name
            plugin_name = getattr(obj, 'name', name)
            plugins[plugin_name] = obj
    
    return plugins

# Discover and list plugins
available_plugins = discover_plugins(Plugin)

print("Discovered plugins:")
for name, plugin_class in available_plugins.items():
    print(f"  - {name}: {plugin_class.__name__}")

# Use the plugins
print("\nExecuting plugins:")
for name, plugin_class in available_plugins.items():
    plugin = plugin_class()
    result = plugin.execute()
    print(f"  {name}: {result}")

# This pattern is used by:
# - Flask blueprints
# - Django apps
# - pytest plugins
# - AI agent tools

# ✅ Expected output:
# Discovered plugins:
#   - email: EmailPlugin
#   - sms: SMSPlugin
#   - slack: SlackPlugin
#
# Executing plugins:
#   email: Sending email...
#   sms: Sending SMS...
#   slack: Posting to Slack...

Metaclasses — Advanced Class Creation

Metaclasses intercept class creation itself, enabling automatic registration and validation.

Descriptors — Custom Attribute Behavior

Descriptors control how attributes are accessed, enabling typed fields and validation.

# Type-checked descriptor
class TypedField:
    """Descriptor that enforces type checking."""
    
    def __init__(self, field_type, default=None):
        self.field_type = field_type
        self.default = default
        self.name = None
    
    def __set_name__(self, owner, name):
        """Called when descriptor is assigned to a class attribute."""
        self.name = name
    
    def __get__(self, instance, owner):
        """Called when attribute is accessed."""
        if instance is None:
            return self
        return instance.__dict__.get(self.name, self.default)
    
    def __set__(self, instance, value):
        """Called when attribute is assigned."""
        if not isinstance(value, self.field_type):
            raise TypeError(
                f"{self.name} must be {self.field_type.__name__}, "
                f"got {type(value).__name__}"
            )
        instance.__dict__[self.name] = value

# Use descriptors for type-safe class
class User:
    """User with type-checked fields."""
    name = TypedField(str, default="Unknown")
    age = TypedField(int, default=0)
    email = TypedField(str)

# Test it
user = User()
print("Setting valid values:")
user.name = "Alice"
user.age = 30
user.email = "[email protected]"
print(f"User: {user.name}, {user.age}, {user.email}")

print("\nTrying invalid value:")
try:
    user.age = "not a number"
except TypeError as e:
    print(f"Error: {e}")

# Descriptors power:
# - @property
# - dataclasses
# - Django ORM fields
# - SQLAlchemy columns
# - Pydantic models

# ✅ Expected output:
# Setting valid values:
# User: Alice, 30, [email protected]
#
# Trying invalid value:
# Error: age must be int, got str

Auto-Generating Documentation

Use introspection to automatically generate documentation from code.

Building a Dynamic API Router

Use introspection to auto-discover and route API endpoints.

Summary

You've mastered Python's metaprogramming and introspection capabilities:

• Basic introspection with dir(), type(), vars()
• The inspect module for deep code analysis
• Inspecting function signatures and parameters
• Detecting object types (functions, classes, coroutines)
• Examining call stacks and frames
• Dynamic function generation and factories
• Intercepting attribute access with __getattr__ and __setattr__
• Inspecting closures and captured variables
• Building plugin systems with auto-discovery
• Metaclasses for controlling class creation
• Descriptors for custom attribute behavior
• Auto-generating documentation from code
• Building dynamic API routers

These techniques power major frameworks like Django, FastAPI, SQLAlchemy, pytest, and Pydantic. Metaprogramming enables you to build flexible, self-documenting systems that adapt to code structure automatically.

🎯 Mini-Challenge: Write Your Own describe()

Debuggers, admin panels and REPL helpers all ship some version of this function: hand it any object and it tells you what the object is, what data it is holding, and what it can do. Write it using introspection only — no hard-coded attribute names anywhere.

📋 Quick Reference — Metaprogramming

🎉 Great work! You've completed this lesson.

You can now write code that introspects and modifies other code at runtime — the power behind pytest, FastAPI, Django ORM, and Pydantic.

Practice quiz

Continue this course

• Previous: Functional Programming Techniques in Python
• Next: Building and Reusing Custom Decorator Libraries — Create your own decorator toolkit with functools.wraps and class decorators
• Quick reference: Python cheat sheet