itertools
Published by Brayan K
itertools is a toolbox of fast, lazy iterator helpers built right into Python. It lets you combine, slice, group, and generate sequences with clean one-liners — and crucially, without loading everything into memory at once.
Part of the free Python course at LearnCodingFast — hands-on lessons with examples you run in your browser, plus practice exercises and a quick quiz.
Every function in this lesson hands back an iterator
An iterator can avoid storing every output, but some helpers keep their inputs: product(), permutations() and combinations() collect finite input pools before producing results. cycle() saves items as it reads them so it can repeat them. Choose the helper with its input storage in mind as well as its output.
An iterator makes its values one at a time, as a loop or list() asks for them, and it's used up after one pass, like the zip objects in enumerate & zip and the round-bracket generators in Comprehensions. The examples import each function by name, as in from itertools import count. count(1001) counts up from 1001 and never stops; islice() takes a few values from it:
from itertools import count, islice
tickets = count(1001)
print(tickets)
print(list(islice(tickets, 3)))
print(list(islice(tickets, 3)))
nums = [1, 2, 3, 4, 5, 6]
print(list(islice(nums, 3)))
print(list(islice(nums, 3)))
# ✅ Expected output:
# count(1001)
# [1001, 1002, 1003]
# [1004, 1005, 1006]
# [1, 2, 3]
# [1, 2, 3]Printing tickets shows count(1001), the recipe rather than the numbers. The second islice() carries on from 1004, because the counter remembers where it got to. A list doesn't work that way: each islice() over nums starts a new pass from the front. Printing most other itertools objects shows only something like <itertools.chain object at 0x7f21d1ec2470>, with an address that changes from run to run.
count() and cycle(), which repeats a sequence, never run out, so something has to stop them: islice(), a break, or zip(), which stops at its shortest input.
from itertools import count, cycle, islice
print(list(islice(count(1), 5)))
print(list(islice(cycle(["red", "green"]), 5)))
for n in count(10, 5):
if n > 25:
break
print(n)
# ✅ Expected output:
# [1, 2, 3, 4, 5]
# ['red', 'green', 'red', 'green', 'red']
# 10
# 15
# 20
# 25from itertools import cycle
players = ["Ada", "Ben", "Cleo", "Dev", "Eve"]
for player, team in zip(players, cycle(["red", "blue"])):
print(player, team)
# ✅ Expected output:
# Ada red
# Ben blue
# Cleo red
# Dev blue
# Eve redlist(count()) on its own never finishes. In this site's editor the page stops responding, as it does for an endless loop in Loops - For and While. Writing your own iterators with yield is Generators & Iterators Mastery.
islice() instead of [start:stop]
islice() takes the same numbers as slicing, a start, a stop and a step, and works on anything a for loop can go through:
from itertools import islice
letters = "abcdefgh"
print(list(islice(letters, 3)))
print(list(islice(letters, 2, 5)))
print(list(islice(letters, 1, None, 2)))
print(list(islice(letters, 5, 2)))
# ✅ Expected output:
# ['a', 'b', 'c']
# ['c', 'd', 'e']
# ['b', 'd', 'f', 'h']
# []None as the stop means "to the end", and a stop before the start gives nothing, as with slicing. Square brackets don't work on an iterator at all, because it has no positions, only a next value:
from itertools import count
tickets = count(1001)
print(tickets[0])
# ✅ Expected error:
# TypeError: 'itertools.count' object is not subscriptableSlicing can also count from the end with negative numbers. islice() can't, because an iterator doesn't know where its end is until it gets there:
from itertools import islice
print(list(islice("abcdefgh", -3, None)))
# ✅ Expected error:
# ValueError: Indices for islice() must be None or an integer: 0 <= x <= sys.maxsize.For text or a list, slice it as usual: "abcdefgh"[-3:] gives fgh.
To work through a list a few items at a time, turn it into an iterator with iter(), which makes one that remembers its place, like tickets above. Then take three at a time, and stop when a batch comes back empty. Iterators: __iter__ & __next__ explains how iter() works.
from itertools import islice
names = ["Ada", "Ben", "Cleo", "Dev", "Eve", "Finn", "Gus"]
queue = iter(names)
batch = list(islice(queue, 3))
while batch:
print(batch)
batch = list(islice(queue, 3))
# ✅ Expected output:
# ['Ada', 'Ben', 'Cleo']
# ['Dev', 'Eve', 'Finn']
# ['Gus']Python 3.12 added batched() to itertools, which does this in one call. This site's editor runs Python 3.11, where from itertools import batched stops with an ImportError, so this lesson sticks to islice().
Joining up: zip_longest() and chain()
zip() stops at the end of the shorter list without a word. zip_longest() goes on to the end of the longest one and fills the gaps with None, or with the fillvalue you give it:
from itertools import zip_longest
names = ["Ada", "Ben", "Cleo"]
scores = [88, 72]
print(list(zip(names, scores)))
print(list(zip_longest(names, scores)))
print(list(zip_longest(names, scores, fillvalue=0)))
# ✅ Expected output:
# [('Ada', 88), ('Ben', 72)]
# [('Ada', 88), ('Ben', 72), ('Cleo', None)]
# [('Ada', 88), ('Ben', 72), ('Cleo', 0)]The fill value has to be given by name. On its own, the 0 is taken as a third thing to zip, and a number isn't something a loop can go through:
from itertools import zip_longest
names = ["Ada", "Ben", "Cleo"]
scores = [88, 72]
print(list(zip_longest(names, scores, 0)))
# ✅ Expected error:
# TypeError: 'int' object is not iterablechain() goes through several inputs one after the other. Give it a list of lists and it goes through that one list, handing back the inner lists unchanged. Put a * in front to pass the inner lists separately, as in *args & **kwargs, or use chain.from_iterable(), which does the same without the star:
from itertools import chain
weeks = [[3, 5], [4], [6, 2]]
print(list(chain(weeks)))
print(list(chain(*weeks)))
print(list(chain.from_iterable(weeks)))
print(sum(chain.from_iterable(weeks)))
# ✅ Expected output:
# [[3, 5], [4], [6, 2]]
# [3, 5, 4, 6, 2]
# [3, 5, 4, 6, 2]
# 20Only one level comes apart: a list inside an inner list stays a list, so chain.from_iterable([[1, [2, 3]], [4]]) gives [1, [2, 3], 4]. The iterator returned by chain() reads each input only when it gets to it, so a mistake can show up after some output:
from itertools import chain
for n in chain([1, 2], 3):
print(n)
# ✅ Expected output:
# 1
# 2
# ✅ Expected error:
# TypeError: 'int' object is not iterabletakewhile() and dropwhile() stop at the first miss
filter() tests every item. takewhile() takes items until the first one that fails the test, then stops. dropwhile() skips items until the first one that fails, then keeps everything after it without testing again. filterfalse() is filter() the other way round. The test is a def here because four calls use it, as Lambda Functions advises:
from itertools import takewhile, dropwhile, filterfalse
def cool(temp):
return temp < 20
temps = [12, 15, 21, 9, 25, 8]
print(list(filter(cool, temps)))
print(list(takewhile(cool, temps)))
print(list(dropwhile(cool, temps)))
print(list(filterfalse(cool, temps)))
# ✅ Expected output:
# [12, 15, 9, 8]
# [12, 15]
# [21, 9, 25, 8]
# [21, 25]dropwhile() kept 9 and 8, which are under 20, because it stopped testing at 21. That makes it the tool for skipping lines at the start, such as the comments at the top of a file, and the wrong tool for removing them everywhere:
from itertools import dropwhile
def is_comment(line):
return line.startswith("#")
log = ["# header", "# version 2", "start", "# note", "stop"]
print(list(dropwhile(is_comment, log)))
print([line for line in log if not is_comment(line)])
# ✅ Expected output:
# ['start', '# note', 'stop']
# ['start', 'stop']accumulate() keeps every step
sum() gives one total. accumulate() gives the running total after every item, and initial= puts a starting value in front, so the result is one longer than the input:
from itertools import accumulate
moves = [20, -5, -30, 10]
balances = list(accumulate(moves, initial=100))
print(balances)
print(len(moves), len(balances))
print(min(balances))
# ✅ Expected output:
# [100, 120, 115, 85, 95]
# 4 5
# 85A function as the second argument accumulates something other than a sum: max gives the highest value so far, and a lambda can multiply.
from itertools import accumulate
sales = [100, 50, 200, 75]
print(list(accumulate(sales)))
print(list(accumulate(sales, max)))
print(list(accumulate([1, 2, 3, 4], lambda total, n: total * n)))
# ✅ Expected output:
# [100, 150, 350, 425]
# [100, 100, 200, 200]
# [1, 2, 6, 24]Because the second argument is the function, accumulate(moves, 100) stops with TypeError: 'int' object is not callable; a starting value goes in initial=. When only the final value matters, reduce() gives just that; functools: lru_cache, partial, reduce covers it.
groupby() only groups neighbours
groupby() walks through the items in order and starts a new group each time the key changes. Items with the same key that aren't next to each other end up in separate groups:
from itertools import groupby
words = ["apple", "avocado", "banana", "apricot", "blueberry"]
for letter, group in groupby(words, key=lambda word: word[0]):
print(letter, list(group))
# ✅ Expected output:
# a ['apple', 'avocado']
# b ['banana']
# a ['apricot']
# b ['blueberry']Sort by the same key first. The key is used twice now, so it's a def:
from itertools import groupby
def first_letter(word):
return word[0]
words = ["apple", "avocado", "banana", "apricot", "blueberry"]
for letter, group in groupby(sorted(words, key=first_letter), key=first_letter):
print(letter, list(group))
# ✅ Expected output:
# a ['apple', 'avocado', 'apricot']
# b ['banana', 'blueberry']Sorting by a different key doesn't help, because groupby() needs the items with equal keys side by side. Each group is an iterator too, used up once it's read, so call list() on it once and keep the result. Groups share the source iterator: advancing the outer groupby() loop makes the previous group unavailable, even if you never read it:
from itertools import groupby
def team_of(player):
return player[0]
for team, members in groupby(["A1", "A2", "B1"], key=team_of):
print(team, len(list(members)), list(members))
# ✅ Expected output:
# A 2 []
# B 1 []When the data already comes in runs, no sort is needed. Counting each run is the start of run-length encoding:
from itertools import groupby
print([(letter, len(list(run))) for letter, run in groupby("aaabbca")])
# ✅ Expected output:
# [('a', 3), ('b', 2), ('c', 1), ('a', 1)]To count or collect items by key without sorting them first, Counter & defaultdict has the tools.
product(), permutations() and combinations()
Four functions list every way to pick items from a sequence. They differ on two questions: does the order matter, and can the same item be picked twice?
from itertools import product, permutations, combinations, combinations_with_replacement
print(["".join(p) for p in product("ABC", repeat=2)])
print(["".join(p) for p in permutations("ABC", 2)])
print(["".join(c) for c in combinations("ABC", 2)])
print(["".join(c) for c in combinations_with_replacement("ABC", 2)])
# ✅ Expected output:
# ['AA', 'AB', 'AC', 'BA', 'BB', 'BC', 'CA', 'CB', 'CC']
# ['AB', 'AC', 'BA', 'BC', 'CA', 'CB']
# ['AB', 'AC', 'BC']
# ['AA', 'AB', 'AC', 'BB', 'BC', 'CC']- product(): order matters and repeats are allowed, so AA and both AB and BA.
- permutations(): order matters, no repeats.
- combinations(): order doesn't matter, no repeats, so AB but not BA.
- combinations_with_replacement(): order doesn't matter, repeats allowed.
These lists grow fast. To know how many there are without making them, the math module counts them: math.perm(n, r) is the number of permutations of r items picked from n, and math.comb(n, r) the number of combinations.
import math
from itertools import product, permutations, combinations
n, r = 5, 3
print(len(list(product(range(n), repeat=r))), n ** r)
print(len(list(permutations(range(n), r))), math.perm(n, r))
print(len(list(combinations(range(n), r))), math.comb(n, r))
print(math.comb(52, 5))
# ✅ Expected output:
# 125 125
# 60 60
# 10 10
# 2598960There are 2,598,960 five-card hands in a 52-card deck, far too many to build a list of just to count them. The functions pick by position, not by value, so combinations([1, 1, 2], 2) gives (1, 2) twice. combinations() also needs to know how many to pick; without it, it stops with TypeError: combinations() missing required argument 'r' (pos 2).
pairwise() gives each item together with the next one, which is how to find the changes between readings:
from itertools import pairwise
readings = [12, 15, 11, 18]
print(list(pairwise(readings)))
print([later - earlier for earlier, later in pairwise(readings)])
# ✅ Expected output:
# [(12, 15), (15, 11), (11, 18)]
# [3, -4, 7]Worked example: orders from two tills
The morning and evening orders arrive as two lists. The shop wants a total per item and a running total of the quantities:
from itertools import chain, groupby, accumulate
morning = [{"item": "tea", "qty": 2}, {"item": "cake", "qty": 1}]
evening = [{"item": "tea", "qty": 3}, {"item": "coffee", "qty": 4}, {"item": "cake", "qty": 2}]
def item_of(order):
return order["item"]
orders = list(chain(morning, evening))
print(len(orders), "orders")
orders.sort(key=item_of)
for item, group in groupby(orders, key=item_of):
print(item, sum(order["qty"] for order in group))
quantities = [order["qty"] for order in orders]
print(quantities)
print(list(accumulate(quantities)))
# ✅ Expected output:
# 5 orders
# cake 3
# coffee 4
# tea 5
# [1, 2, 4, 2, 3]
# [1, 3, 7, 9, 12]orders.sort() rearranged the list itself, which is why quantities comes out in item order. If the original order matters later, group sorted(orders, key=item_of) instead, which leaves orders as it was.
Practice: totals per day
Fill the three blanks with the values in the hint lines, in order, so that the sales are totalled per day and then as a running total.
from itertools import groupby, accumulate
def day_of(sale):
return sale[0]
sales = [("Mon", 5), ("Tue", 3), ("Mon", 2), ("Tue", 6), ("Wed", 1)]
day_totals = []
for day, rows in groupby(___(sales, key=day_of), key=___):
total = sum(sale[1] for sale in rows)
day_totals.append(total)
print(day, total)
print("Running:", list(___(day_totals)))
# 👉 replace ___ with sorted
# 👉 replace ___ with day_of
# 👉 replace ___ with accumulate
# ✅ Expected output:
# Mon 7
# Tue 9
# Wed 1
# Running: [7, 16, 17]On your own: matches on two courts
Ada, Ben, Cleo and Dev each play every other player once, on two courts in turn. Print each match with its court, such as Court 1: Ada v Ben, then the number of matches, checked against math.comb(). Then compare with this solution:
from itertools import combinations, cycle
import math
players = ["Ada", "Ben", "Cleo", "Dev"]
matches = list(combinations(players, 2))
for (home, away), court in zip(matches, cycle(["Court 1", "Court 2"])):
print(f"{court}: {home} v {away}")
print(len(matches), "matches, and math.comb says", math.comb(len(players), 2))
# ✅ Expected output:
# Court 1: Ada v Ben
# Court 2: Ada v Cleo
# Court 1: Ada v Dev
# Court 2: Ben v Cleo
# Court 1: Ben v Dev
# Court 2: Cleo v Dev
# 6 matches, and math.comb says 6combinations() makes each pairing once, cycle() alternates the courts for as long as zip() asks, and (home, away), court unpacks a pair inside a pair, as in enumerate & zip.
Practice quiz
What does list(dropwhile(lambda t: t < 20, [12, 15, 21, 9, 25, 8])) give?
- [21, 9, 25, 8]
- [12, 15]
- [21, 25]
- [12, 15, 9, 8]
Answer: [21, 9, 25, 8]. dropwhile() skips items only until the first one that fails the test, 21. After that it keeps everything, 9 and 8 included.
What does [(k, len(list(g))) for k, g in groupby("aaabbca")] give?
- [('a', 4), ('b', 2), ('c', 1)]
- [('a', 3), ('b', 2), ('c', 1)]
- [('a', 3), ('b', 2), ('c', 1), ('a', 1)]
- An error
Answer: [('a', 3), ('b', 2), ('c', 1), ('a', 1)]. groupby() starts a new group whenever the key changes, so the last a is a group of its own.
groups = list(groupby("aaabb")). What does list(groups[0][1]) give?
- ['a', 'a', 'a']
- []
- ['a']
- An error
Answer: []. list() moved groupby() past the first group, so that group's items were used up. Read each group inside the loop.
What does list(chain([[1, 2], [3]])) give?
- [1, 2, 3]
- [[1, 2, 3]]
- An error
- [[1, 2], [3]]
Answer: [[1, 2], [3]]. chain() got one list and went through it, handing back its two items, the inner lists. chain(*lists) or chain.from_iterable(lists) joins them.
What does list(accumulate([20, -5, -30], initial=100)) give?
- [120, 115, 85]
- [100, 120, 115, 85]
- [85]
- [20, 15, -15]
Answer: [100, 120, 115, 85]. initial= puts 100 first and each step adds the next move, so the result is one longer than the input.
What does list(islice("abcdefgh", -3, None)) do?
- It stops with ValueError: Indices for islice() must be None or an integer: 0 <= x <= sys.maxsize.
- ['f', 'g', 'h']
- []
- ['a', 'b', 'c', 'd', 'e']
Answer: It stops with ValueError: Indices for islice() must be None or an integer: 0 <= x <= sys.maxsize.. islice() can't count from the end, because an iterator doesn't know where its end is. For text or a list, slice it: "abcdefgh"[-3:].
How many items do combinations("ABCD", 2) and permutations("ABCD", 2) give?
- 12 and 6
- 6 and 6
- 4 and 8
- 6 and 12
Answer: 6 and 12. combinations() ignores order, so AB and BA are one pick: 6. permutations() counts them separately: 12.
What does list(zip_longest("abc", [1, 2], fillvalue=0)) give?
- [('a', 1), ('b', 2)]
- [('a', 1), ('b', 2), ('c', None)]
- [('a', 1), ('b', 2), ('c', 0)]
- An error
Answer: [('a', 1), ('b', 2), ('c', 0)]. zip_longest() goes on to the end of the longer input and puts the fillvalue in the gaps.
Continue this course
- Previous: Web Scraping with BeautifulSoup
- Next: Modern File Paths with pathlib — Handle file paths cleanly with the modern pathlib module
- Quick reference: Python cheat sheet › Standard Library
Frequently asked questions
Is itertools faster than writing the loop myself?
Sometimes. CPython's itertools is written in C, but much of a real program's time goes on what you do with each item. Returning an iterator can avoid building every result at once, but does not guarantee constant memory: product, permutations and combinations keep input pools, and cycle caches items as it reads them. If speed or memory matters, measure your case. Benchmarking with timeit covers timing.
Where are izip() and imap()?
They were Python 2's lazy versions of zip() and map(). In Python 3, zip() and map() are lazy themselves, so itertools dropped the extra names: from itertools import izip stops with an ImportError.
How do I get the last few items of an iterator?
islice() can't count from the end. If the values fit in memory, make a list and slice it: list(values)[-3:]. To keep only the last few while going through a long iterator, collections.deque(values, maxlen=3) holds the newest three and drops the rest.