multiprocessing: True Parallelism

The multiprocessing module runs your code in separate OS processes — each with its own Python interpreter and its own GIL — so CPU-heavy work can run on multiple cores at the same time.

Learn multiprocessing: True Parallelism in our free Python course — an interactive lesson with runnable examples, a practice exercise and a quick reference.

Part of the free Python course at LearnCodingFast — hands-on lessons with examples you run in your browser, plus practice exercises and a quick quiz.

Where threads take turns under the GIL, processes run genuinely in parallel. This is the tool you reach for when you need raw computational throughput.

A Process works much like a Thread — give it a target and args , then start() and join() . But because processes don't share memory, you can't just write to a shared dict. Instead you pass results back through a Queue . Notice the if __name__ == "__main__": guard — it's mandatory.

A Pool manages a fixed set of worker processes. pool.map(func, iterable) splits the iterable across the workers, runs them in parallel on different cores , and returns the results in order . This is the cleanest way to parallelize CPU-bound work:

Because each heavy call runs in its own process with its own GIL, four cores can grind away simultaneously — something threads simply cannot do for this kind of pure-Python loop.

concurrent.futures.ProcessPoolExecutor is the modern, unified interface. The best part: it shares the exact same API as ThreadPoolExecutor from the previous lesson. Swap one class name and your code goes from threads to processes:

Replace each ___ so a Pool doubles every number. Remember the guard and which method maps a function over the list.

If each task is trivial, process startup and pickling overhead can make multiprocessing slower than a plain loop. Reserve it for genuinely heavy, CPU-bound work.

Use a ProcessPoolExecutor to count the vowels in each word, build a word→count dictionary, and print the total number of vowels.

Lesson complete — you can use every CPU core!

You can launch a Process , return data through a Queue , fan work out with Pool.map , and use the modern ProcessPoolExecutor . You understand why processes sidestep the GIL for CPU-bound work — and you never forget the if __name__ == "__main__": guard.

🚀 Up next: queue — thread-safe queues for clean producer/consumer pipelines.

Practice quiz

How does the multiprocessing module achieve true parallelism despite the GIL?

  • It disables the GIL for all threads
  • It runs everything on a single core faster
  • Each process is a separate interpreter with its own GIL, so the OS can run them on different cores
  • It converts processes into threads

Answer: Each process is a separate interpreter with its own GIL, so the OS can run them on different cores. Separate processes each have their own interpreter and GIL, so they truly run in parallel on multiple cores.

Why is the 'if __name__ == "__main__":' guard required for multiprocessing?

  • Without it, spawning processes on Windows/macOS re-imports the script and recursively launches more processes
  • It speeds up imports
  • It is only needed for threads
  • It silences warnings

Answer: Without it, spawning processes on Windows/macOS re-imports the script and recursively launches more processes. Process spawning re-imports your module; the guard ensures launch code runs only in the original program.

What does pool.map(func, iterable) return?

  • A single combined value
  • Results in random completion order
  • Nothing; it only has side effects
  • The results in the same order as the input iterable

Answer: The results in the same order as the input iterable. Pool.map splits work across workers but returns results ordered to match the input.

Because processes do NOT share memory, how do you get results back from a Process?

  • By modifying a global variable
  • Through a Queue (or similar IPC) that the process puts results into
  • By printing them only
  • Results are shared automatically

Answer: Through a Queue (or similar IPC) that the process puts results into. Each process has its own memory, so results are passed back via a Queue, pipe, or return value.

Multiprocessing is the BEST fit for which kind of workload?

  • CPU-bound tasks like heavy math or image processing
  • I/O-bound tasks that mostly wait
  • Tasks that need shared mutable state
  • Very tiny, trivial tasks

Answer: CPU-bound tasks like heavy math or image processing. Processes shine for CPU-bound work because they use multiple cores; threads are better for I/O-bound waiting.

Which modern class shares the same API as ThreadPoolExecutor but uses processes?

  • multiprocessing.Manager
  • asyncio.ProcessPool
  • concurrent.futures.ProcessPoolExecutor
  • threading.ProcessPool

Answer: concurrent.futures.ProcessPoolExecutor. ProcessPoolExecutor mirrors ThreadPoolExecutor's API, so you can switch engines by changing one class name.

Why might trying to update a shared global variable from a child process NOT work?

  • Globals are read-only in Python
  • The child modifies its own copy of the variable, not the parent's
  • Multiprocessing deletes globals
  • It always works

Answer: The child modifies its own copy of the variable, not the parent's. Processes don't share memory, so the child changes its own copy; you must return values and combine them.

When can multiprocessing actually be SLOWER than a plain loop?

  • When each task is heavy and CPU-bound
  • When you use a Pool
  • When you use the __main__ guard
  • When tasks are tiny and fast, so startup and pickling overhead dominates

Answer: When tasks are tiny and fast, so startup and pickling overhead dominates. Process startup and pickling cost can outweigh the work for trivial tasks, making it slower.

What does the pattern [Process(target=square, args=(n, q)) for n in nums] create?

  • One process that loops over nums
  • A list of Process objects, one per value of n
  • A thread pool
  • A Queue of results

Answer: A list of Process objects, one per value of n. It builds one Process per item; you then start() and join() each of them.

Why must data sent to and from worker processes be picklable?

  • To compress it
  • To encrypt it
  • Because processes don't share memory, so arguments and results are serialized (pickled) and copied
  • Because the GIL requires it

Answer: Because processes don't share memory, so arguments and results are serialized (pickled) and copied. Crossing process boundaries means data is pickled and copied, so non-picklable objects fail.