Checkpoint: Concurrency & I/O

A checkpoint is a consolidation lesson where you recap and combine the skills from a whole module — here, the eight tools that let Python do many things at once and work with files and the network.

Learn Checkpoint: Concurrency & I/O in our free Python course — an interactive lesson with runnable examples, a practice exercise and a quick reference.

Part of the free Python course at LearnCodingFast — hands-on lessons with examples you run in your browser, plus practice exercises and a quick quiz.

You'll review threading, the GIL, multiprocessing, queues, contextlib, tempfile, glob, shutil, and sockets, then prove it all with a build challenge and a quick quiz.

Before the challenge, here is the whole module in one table — what each tool is for and the one thing to remember about it:

Put it all together. A list of orders must be processed concurrently with a ThreadPoolExecutor , each result written into a TemporaryDirectory , and the totals aggregated deterministically . The starter has a couple of ___ blanks for you to fill — the full solution follows below.

The blanks are pool.map (maps the function over every order, in parallel) and sorted (gives stable, deterministic output regardless of which thread finishes first):

Why it works: the executor runs process on every order concurrently and returns the results in input order. We persist each total to its own file in a self-cleaning temp dir, sum for the grand total, and sorted() the pairs so the printed summary is identical on every run — concurrency without nondeterminism.

1. You have 100 URLs to download. Threads or processes — and why?

Threads (a ThreadPoolExecutor ). Downloading is I/O-bound — the program mostly waits on the network, and the GIL is released during that wait, so threads overlap the waiting beautifully. Processes would add overhead for no gain here.

2. Why does pure-Python CPU work not speed up with threads?

Because of the GIL : only one thread runs Python bytecode at a time, so CPU-bound threads just take turns on one core. Use multiprocessing / ProcessPoolExecutor for true multi-core parallelism.

3. You start 4 consumer threads that loop on q.get() . How many poison pills (None) must you send to stop them all?

Four — one per worker. Each thread consumes exactly one None and then breaks out of its loop. Send fewer and some threads never stop, so join() on them hangs.

4. What is wrong with shutil.copy("my_folder", "dest") ?

copy only handles single files; on a directory it raises IsADirectoryError . Use shutil.copytree("my_folder", "dest") to copy a folder and its contents recursively.

5. Why must you call .encode() before sock.sendall(...) ?

Sockets send bytes , not str . .encode() converts a string to bytes (UTF-8 by default) for sending; on the other side you call .decode() to turn the received bytes back into a string.

6. Why wrap demos and tests in a TemporaryDirectory ?

It gives a safe, unique, platform-correct location that cleans itself up when the with block ends — no leftover files, no hardcoded /tmp , and no risk of clobbering real data.

Checkpoint cleared — you handle concurrency & I/O!

You can pick threads vs processes by workload, build producer/consumer pipelines with queues, write clean context managers, work with temp files and whole directory trees safely, find files by pattern, and talk over the network with sockets. That's a complete, professional toolkit for "doing many things at once."

🚀 Up next: the statistics module — mean, median, and friends from the standard library.

Practice quiz

You need to download 100 URLs. Which tool fits best, and why?

  • multiprocessing — downloads are CPU-bound
  • A single thread — concurrency would not help
  • A ThreadPoolExecutor — downloads are I/O-bound
  • asyncio is the only option that works

Answer: A ThreadPoolExecutor — downloads are I/O-bound. Downloading is I/O-bound: the program mostly waits on the network, and the GIL is released during the wait, so threads overlap nicely.

Why does pure-Python CPU-bound work NOT speed up with threads?

  • The GIL lets only one thread run Python bytecode at a time
  • Threads are slower than processes to start
  • Python has no threading support
  • CPU work always blocks the network

Answer: The GIL lets only one thread run Python bytecode at a time. The Global Interpreter Lock allows only one thread to execute Python bytecode at once, so CPU-bound threads just take turns on one core.

For true multi-core parallelism on CPU-bound work, which module should you use?

  • threading
  • queue
  • contextlib
  • multiprocessing

Answer: multiprocessing. multiprocessing (or a ProcessPoolExecutor) spawns separate processes, each with its own interpreter and GIL, achieving real parallelism across cores.

You run 4 consumer threads looping on q.get(). How many poison pills (None) must you send to stop them all?

  • 1
  • 4
  • 2
  • It depends on the queue size

Answer: 4. Each thread consumes exactly one None and then breaks, so you need one poison pill per worker — four in total.

In queue.Queue, what does task_done() pair with?

  • join()
  • put()
  • get_nowait()
  • empty()

Answer: join(). Each completed item should be acknowledged with task_done(); join() blocks until every put item has had a matching task_done().

A function decorated with @contextlib.contextmanager must yield how many times?

  • Zero times
  • Twice
  • Exactly once
  • Once per item processed

Answer: Exactly once. The generator yields exactly once: code before yield is setup, code after yield is cleanup, and the yielded value is bound by the with statement.

Why wrap demos and tests in a tempfile.TemporaryDirectory()?

  • It runs the code faster
  • It gives a safe, unique path that cleans itself up when the with block ends
  • It is required to import other modules
  • It enables multiprocessing

Answer: It gives a safe, unique path that cleans itself up when the with block ends. A TemporaryDirectory provides a unique, platform-correct location that is deleted automatically on block exit — no leftover files and no hardcoded paths.

What is wrong with shutil.copy("my_folder", "dest") when my_folder is a directory?

  • Nothing — it copies the folder recursively
  • It silently copies an empty folder
  • It moves the folder instead of copying it
  • copy() only handles single files and raises IsADirectoryError

Answer: copy() only handles single files and raises IsADirectoryError. shutil.copy works on single files only; for a directory tree you must use shutil.copytree to copy it and its contents recursively.

Why call .encode() before sock.sendall(...)?

  • To compress the data
  • Because sockets transmit bytes, not str
  • To make the socket non-blocking
  • It is optional and only for speed

Answer: Because sockets transmit bytes, not str. Sockets send and receive bytes. .encode() converts a str to bytes (UTF-8 by default) for sending; the receiver calls .decode() to get a string back.

After collecting filenames with glob, what is recommended for deterministic output?

  • Nothing — glob already returns a sorted list
  • Reverse the list
  • Call sort() on the results, since glob order is not guaranteed
  • Convert the list to a set

Answer: Call sort() on the results, since glob order is not guaranteed. glob does not guarantee any particular order, so sorting the results yourself makes the output stable and reproducible across runs and platforms.