Binary Data with struct
Published by Brayan K
The struct module converts between Python values and compact C-style bytes, letting you pack numbers into binary and unpack raw bytes from files, packets, and devices.
Part of the free Python course at LearnCodingFast — hands-on lessons with examples you run in your browser, plus practice exercises and a quick quiz.
A binary layout must define more than the values
Two programs can agree on the numbers and still disagree on their bytes. A binary contract needs field order, field widths, signedness, byte order and padding rules. struct applies that layout; it does not discover a format from a byte string or verify that the decoded values make sense for your application.
from struct import Struct
record = Struct("!BH")
packed = record.pack(2, 513)
print(record.size)
print(packed.hex())
print(record.unpack(packed))
# ✅ Expected output:
# 3
# 020201
# (2, 513)The exclamation mark selects network byte order: big-endian, standard sizes and no automatic alignment padding. B is a one-byte unsigned integer and H a two-byte unsigned integer. The returned bytes have a fixed layout, while unpack returns a tuple of Python values. Reusing a Struct object keeps the chosen format and size together.
from struct import pack, unpack
big = pack(">H", 513)
little = pack("<H", 513)
print(big.hex())
print(little.hex())
print(unpack(">H", big)[0])
print(unpack("<H", big)[0])
# ✅ Expected output:
# 0201
# 0102
# 513
# 258Reading the same bytes with the wrong byte order can produce another valid integer without raising. The error is a mismatched contract. A missing prefix means native mode, which follows the machine’s C sizes and alignment. The equals prefix keeps standard widths and disables alignment padding but still uses native byte order. Choose an explicit less-than, greater-than or exclamation prefix for a portable external format.
Integer ranges and signedness are enforced
from struct import pack, unpack
data = pack("!b", -1)
print(data.hex())
print(unpack("!b", data)[0])
print(unpack("!B", data)[0])
# ✅ Expected output:
# ff
# -1
# 255from struct import pack, error
try:
pack("!B", 256)
except error as problem:
print(type(problem).__module__ + "." + type(problem).__name__)
# ✅ Expected output:
# struct.errorThe supported Python versions phrase this range error differently. This example reports the actual exception class while the other error examples show their stable exact messages.
A one-byte unsigned field can represent values from zero through 255. A one-byte signed field has a different range. Choosing a larger Python integer does not expand the field in the binary format. Validate your schema before packing and handle struct.error as a failed conversion, not a request to truncate bits.
For real-valued fields, standard f and d formats encode binary floating-point values. Many decimal fractions are approximate in that representation. Numbers: int, float, Decimal & Fraction owns the precision tradeoff; changing the byte order does not make a decimal value exact.
A string field is one fixed-width bytes value
from struct import pack, unpack
print(pack("!4s", b"AB").hex())
print(pack("!4s", b"ABCDE").hex())
print(unpack("!4s", b"AB\x00\x00"))
print(pack("!2c", b"A", b"B").hex())
# ✅ Expected output:
# 41420000
# 41424344
# (b'AB\x00\x00',)
# 41424s means one four-byte field, not four separate arguments. Short values are padded with zero bytes and long values are truncated. In contrast, 2c means two one-byte character fields and therefore two arguments. Fixed-width bytes are not automatically text; encoding and permitted padding must be defined by the surrounding format.
from struct import pack
pack("!4s", "AB")
# ✅ Expected error:
# struct.error: argument for 's' must be a bytes objectDo not blindly strip trailing zero bytes from arbitrary binary data: they may be meaningful payload. If your schema defines a zero-padded text field, removing padding and decoding is a schema decision. Strings, Bytes & Unicode Encodings owns encoding and decoding, including characters occupying multiple bytes.
unpack consumes an exact record; unpack_from reads at an offset
from struct import unpack
unpack("!H", b"\x00\x05extra")
# ✅ Expected error:
# struct.error: unpack requires a buffer of 2 bytesfrom struct import Struct
field = Struct("!H")
buffer = b"AB" + field.pack(513) + b"CD"
print(field.unpack_from(buffer, 2))
print(buffer[:2].decode("ascii"))
print(buffer[-2:].decode("ascii"))
# ✅ Expected output:
# (513,)
# AB
# CDunpack requires exactly the format’s size. unpack_from permits surrounding bytes and starts at the chosen offset, but still requires enough bytes for the record. The offset counts bytes. An offset into the middle of another field can decode successfully yet mean the wrong thing, so validate the packet structure before interpreting offsets.
from struct import Struct
field = Struct("!H")
buffer = bytearray(b"AA....ZZ")
field.pack_into(buffer, 2, 513)
print(buffer.hex())
print(field.unpack_from(buffer, 2))
# ✅ Expected output:
# 414102012e2e5a5a
# (513,)pack_into modifies a writable buffer; it does not grow it to make room. Allocate the complete size first. Immutable bytes can be read by unpack_from but cannot be a writable destination. Padding format x explicitly consumes a byte without consuming a Python argument, which is useful when the schema includes reserved space.
Repeated records still need a complete final record
from struct import Struct, iter_unpack
record = Struct("!Hh")
data = b"".join(record.pack(*row) for row in [(1, -5), (2, 12)])
print(list(iter_unpack(record.format, data)))
print(len(data) // record.size)
# ✅ Expected output:
# [(1, -5), (2, 12)]
# 2from struct import iter_unpack
list(iter_unpack("!H", b"\x00\x01\x02"))
# ✅ Expected error:
# struct.error: iterative unpacking requires a buffer of a multiple of 2 bytesiter_unpack returns an iterator of tuples and requires a nonzero record size with a buffer length that is an exact multiple of that size. It is a parser for a complete supplied buffer, not a streaming network receiver. Network Programming with sockets owns collecting partial receives and defining message boundaries.
Successful decoding is not semantic validation
from struct import Struct
header = Struct("!BB")
version, count = header.unpack(header.pack(9, 2))
if version != 1:
raise ValueError("unsupported record version")
# ✅ Expected error:
# ValueError: unsupported record versionstruct can prove that bytes fit a layout; it cannot know your allowed version numbers, ranges, checksums or relationships between fields. A count also needs a sensible limit before it controls allocation or iteration. Check these application rules after decoding the header and before trusting its body.
Complete a portable two-field record
Use a big-endian unsigned byte followed by an unsigned short. Pack the values 3 and 1025, then unpack them using the same Struct object. The hints select the format and the decoding method.
from struct import Struct
record = Struct(___)
data = record.pack(3, 1025)
print(data.hex())
print(record.___(data))
# 👉 replace ___ with "!BH"
# 👉 replace ___ with unpack
# ✅ Expected output:
# 030401
# (3, 1025)Validate a counted packet before reading its records
Define a packet with a one-byte version, a two-byte count and that many signed two-byte readings. Version must be 1, count must be at most 64, and the packet length must match exactly. Encode the readings -15 and 230, decode them into a list, and reject a truncated copy with a clear application error. The complete data is local bytes, with no network operation.
from struct import Struct
HEADER = Struct("!BH")
READING = Struct("!h")
def decode_packet(packet):
if len(packet) < HEADER.size:
raise ValueError("incomplete header")
version, count = HEADER.unpack_from(packet)
if version != 1:
raise ValueError("unsupported version")
if count > 64:
raise ValueError("too many readings")
expected_size = HEADER.size + count * READING.size
if len(packet) != expected_size:
raise ValueError("packet length does not match count")
return [READING.unpack_from(packet, HEADER.size + index * READING.size)[0]
for index in range(count)]
values = [-15, 230]
packet = HEADER.pack(1, len(values))
packet += b"".join(READING.pack(value) for value in values)
print(decode_packet(packet))
try:
decode_packet(packet[:-1])
except ValueError as error:
print(error)
# ✅ Expected output:
# [-15, 230]
# packet length does not match count
Practice quiz
What does this program print? from struct import pack print(pack("!H", 513).hex())
- 0201
- 0102
- 02010000
- 513
Answer: 0201. Big-endian places the most significant byte first in a two-byte field.
What does this program print? from struct import unpack print(unpack("!B", b"\xff"))
- -1
- (255,)
- 255
- (65535,)
Answer: (255,). An unsigned one-byte field decodes to 255, and unpack returns a tuple.
Which format prefix still uses native byte order despite standard field sizes?
- !
- >
- =
- <
Answer: =. The equals prefix removes native sizes/alignment but retains native byte order.
What does this program print? from struct import pack print(pack("!3s", b"ABCDE").hex())
- 4142434445
- 41424300
- 0300414243
- 414243
Answer: 414243. An s field truncates the supplied bytes to its fixed width.
What does this program print? from struct import calcsize print(calcsize("!BH"))
- 3
- 4
- 8
- 2
Answer: 3. Standard B and H fields occupy one and two bytes without automatic padding.
Which operation can read a record from the middle of a larger buffer?
- unpack with any extra bytes
- unpack_from with an explicit offset
- pack_into on immutable bytes
- Decoding the entire buffer as UTF-8
Answer: unpack_from with an explicit offset. unpack_from reads the format-sized record at a byte offset.
A header unpacks successfully with version 9, but your schema only accepts version 1. What follows?
- The version is valid because unpack succeeded
- The count must be ignored
- Application validation must reject the unsupported version
- struct automatically upgrades the schema
Answer: Application validation must reject the unsupported version. Structural decoding cannot enforce application meaning.
What does this program print? from struct import iter_unpack print(list(iter_unpack("!B", b"\x01\x02")))
- [1, 2]
- (1, 2)
- [(1, 2)]
- [(1,), (2,)]
Answer: [(1,), (2,)]. Each fixed-size record yields its own tuple.
Continue this course
- Previous: The operator Module
- Next: Text Wrapping & Formatting (textwrap) — Wrap, indent, dedent and shorten text cleanly with textwrap
- Quick reference: Python cheat sheet