Binary Data with struct

Published by Brayan K

The struct module converts between Python values and compact C-style bytes, letting you pack numbers into binary and unpack raw bytes from files, packets, and devices.

Part of the free Python course at LearnCodingFast — hands-on lessons with examples you run in your browser, plus practice exercises and a quick quiz.

A binary layout must define more than the values

Two programs can agree on the numbers and still disagree on their bytes. A binary contract needs field order, field widths, signedness, byte order and padding rules. struct applies that layout; it does not discover a format from a byte string or verify that the decoded values make sense for your application.

from struct import Struct

record = Struct("!BH")
packed = record.pack(2, 513)
print(record.size)
print(packed.hex())
print(record.unpack(packed))

# ✅ Expected output:
# 3
# 020201
# (2, 513)

The exclamation mark selects network byte order: big-endian, standard sizes and no automatic alignment padding. B is a one-byte unsigned integer and H a two-byte unsigned integer. The returned bytes have a fixed layout, while unpack returns a tuple of Python values. Reusing a Struct object keeps the chosen format and size together.

from struct import pack, unpack

big = pack(">H", 513)
little = pack("<H", 513)
print(big.hex())
print(little.hex())
print(unpack(">H", big)[0])
print(unpack("<H", big)[0])

# ✅ Expected output:
# 0201
# 0102
# 513
# 258

Reading the same bytes with the wrong byte order can produce another valid integer without raising. The error is a mismatched contract. A missing prefix means native mode, which follows the machine’s C sizes and alignment. The equals prefix keeps standard widths and disables alignment padding but still uses native byte order. Choose an explicit less-than, greater-than or exclamation prefix for a portable external format.

Integer ranges and signedness are enforced

from struct import pack, unpack

data = pack("!b", -1)
print(data.hex())
print(unpack("!b", data)[0])
print(unpack("!B", data)[0])

# ✅ Expected output:
# ff
# -1
# 255
from struct import pack, error

try:
    pack("!B", 256)
except error as problem:
    print(type(problem).__module__ + "." + type(problem).__name__)

# ✅ Expected output:
# struct.error

The supported Python versions phrase this range error differently. This example reports the actual exception class while the other error examples show their stable exact messages.

A one-byte unsigned field can represent values from zero through 255. A one-byte signed field has a different range. Choosing a larger Python integer does not expand the field in the binary format. Validate your schema before packing and handle struct.error as a failed conversion, not a request to truncate bits.

For real-valued fields, standard f and d formats encode binary floating-point values. Many decimal fractions are approximate in that representation. Numbers: int, float, Decimal & Fraction owns the precision tradeoff; changing the byte order does not make a decimal value exact.

A string field is one fixed-width bytes value

from struct import pack, unpack

print(pack("!4s", b"AB").hex())
print(pack("!4s", b"ABCDE").hex())
print(unpack("!4s", b"AB\x00\x00"))
print(pack("!2c", b"A", b"B").hex())

# ✅ Expected output:
# 41420000
# 41424344
# (b'AB\x00\x00',)
# 4142

4s means one four-byte field, not four separate arguments. Short values are padded with zero bytes and long values are truncated. In contrast, 2c means two one-byte character fields and therefore two arguments. Fixed-width bytes are not automatically text; encoding and permitted padding must be defined by the surrounding format.

from struct import pack

pack("!4s", "AB")

# ✅ Expected error:
# struct.error: argument for 's' must be a bytes object

Do not blindly strip trailing zero bytes from arbitrary binary data: they may be meaningful payload. If your schema defines a zero-padded text field, removing padding and decoding is a schema decision. Strings, Bytes & Unicode Encodings owns encoding and decoding, including characters occupying multiple bytes.

unpack consumes an exact record; unpack_from reads at an offset

from struct import unpack

unpack("!H", b"\x00\x05extra")

# ✅ Expected error:
# struct.error: unpack requires a buffer of 2 bytes
from struct import Struct

field = Struct("!H")
buffer = b"AB" + field.pack(513) + b"CD"
print(field.unpack_from(buffer, 2))
print(buffer[:2].decode("ascii"))
print(buffer[-2:].decode("ascii"))

# ✅ Expected output:
# (513,)
# AB
# CD

unpack requires exactly the format’s size. unpack_from permits surrounding bytes and starts at the chosen offset, but still requires enough bytes for the record. The offset counts bytes. An offset into the middle of another field can decode successfully yet mean the wrong thing, so validate the packet structure before interpreting offsets.

from struct import Struct

field = Struct("!H")
buffer = bytearray(b"AA....ZZ")
field.pack_into(buffer, 2, 513)
print(buffer.hex())
print(field.unpack_from(buffer, 2))

# ✅ Expected output:
# 414102012e2e5a5a
# (513,)

pack_into modifies a writable buffer; it does not grow it to make room. Allocate the complete size first. Immutable bytes can be read by unpack_from but cannot be a writable destination. Padding format x explicitly consumes a byte without consuming a Python argument, which is useful when the schema includes reserved space.

Repeated records still need a complete final record

from struct import Struct, iter_unpack

record = Struct("!Hh")
data = b"".join(record.pack(*row) for row in [(1, -5), (2, 12)])
print(list(iter_unpack(record.format, data)))
print(len(data) // record.size)

# ✅ Expected output:
# [(1, -5), (2, 12)]
# 2
from struct import iter_unpack

list(iter_unpack("!H", b"\x00\x01\x02"))

# ✅ Expected error:
# struct.error: iterative unpacking requires a buffer of a multiple of 2 bytes

iter_unpack returns an iterator of tuples and requires a nonzero record size with a buffer length that is an exact multiple of that size. It is a parser for a complete supplied buffer, not a streaming network receiver. Network Programming with sockets owns collecting partial receives and defining message boundaries.

Successful decoding is not semantic validation

from struct import Struct

header = Struct("!BB")
version, count = header.unpack(header.pack(9, 2))
if version != 1:
    raise ValueError("unsupported record version")

# ✅ Expected error:
# ValueError: unsupported record version

struct can prove that bytes fit a layout; it cannot know your allowed version numbers, ranges, checksums or relationships between fields. A count also needs a sensible limit before it controls allocation or iteration. Check these application rules after decoding the header and before trusting its body.

Complete a portable two-field record

Use a big-endian unsigned byte followed by an unsigned short. Pack the values 3 and 1025, then unpack them using the same Struct object. The hints select the format and the decoding method.

from struct import Struct

record = Struct(___)
data = record.pack(3, 1025)
print(data.hex())
print(record.___(data))

# 👉 replace ___ with "!BH"
# 👉 replace ___ with unpack

# ✅ Expected output:
# 030401
# (3, 1025)

Validate a counted packet before reading its records

Define a packet with a one-byte version, a two-byte count and that many signed two-byte readings. Version must be 1, count must be at most 64, and the packet length must match exactly. Encode the readings -15 and 230, decode them into a list, and reject a truncated copy with a clear application error. The complete data is local bytes, with no network operation.

from struct import Struct

HEADER = Struct("!BH")
READING = Struct("!h")

def decode_packet(packet):
    if len(packet) < HEADER.size:
        raise ValueError("incomplete header")
    version, count = HEADER.unpack_from(packet)
    if version != 1:
        raise ValueError("unsupported version")
    if count > 64:
        raise ValueError("too many readings")
    expected_size = HEADER.size + count * READING.size
    if len(packet) != expected_size:
        raise ValueError("packet length does not match count")
    return [READING.unpack_from(packet, HEADER.size + index * READING.size)[0]
            for index in range(count)]

values = [-15, 230]
packet = HEADER.pack(1, len(values))
packet += b"".join(READING.pack(value) for value in values)
print(decode_packet(packet))
try:
    decode_packet(packet[:-1])
except ValueError as error:
    print(error)

# ✅ Expected output:
# [-15, 230]
# packet length does not match count

Practice quiz

What does this program print? from struct import pack print(pack("!H", 513).hex())

  • 0201
  • 0102
  • 02010000
  • 513

Answer: 0201. Big-endian places the most significant byte first in a two-byte field.

What does this program print? from struct import unpack print(unpack("!B", b"\xff"))

  • -1
  • (255,)
  • 255
  • (65535,)

Answer: (255,). An unsigned one-byte field decodes to 255, and unpack returns a tuple.

Which format prefix still uses native byte order despite standard field sizes?

  • !
  • >
  • =
  • <

Answer: =. The equals prefix removes native sizes/alignment but retains native byte order.

What does this program print? from struct import pack print(pack("!3s", b"ABCDE").hex())

  • 4142434445
  • 41424300
  • 0300414243
  • 414243

Answer: 414243. An s field truncates the supplied bytes to its fixed width.

What does this program print? from struct import calcsize print(calcsize("!BH"))

  • 3
  • 4
  • 8
  • 2

Answer: 3. Standard B and H fields occupy one and two bytes without automatic padding.

Which operation can read a record from the middle of a larger buffer?

  • unpack with any extra bytes
  • unpack_from with an explicit offset
  • pack_into on immutable bytes
  • Decoding the entire buffer as UTF-8

Answer: unpack_from with an explicit offset. unpack_from reads the format-sized record at a byte offset.

A header unpacks successfully with version 9, but your schema only accepts version 1. What follows?

  • The version is valid because unpack succeeded
  • The count must be ignored
  • Application validation must reject the unsupported version
  • struct automatically upgrades the schema

Answer: Application validation must reject the unsupported version. Structural decoding cannot enforce application meaning.

What does this program print? from struct import iter_unpack print(list(iter_unpack("!B", b"\x01\x02")))

  • [1, 2]
  • (1, 2)
  • [(1, 2)]
  • [(1,), (2,)]

Answer: [(1,), (2,)]. Each fixed-size record yields its own tuple.

Continue this course

Related lessons