Running Commands with subprocess
Published by Brayan K
The subprocess module lets your Python program launch external commands, capture their output and error text, and check whether they succeeded.
Part of the free Python course at LearnCodingFast — hands-on lessons with examples you run in your browser, plus practice exercises and a quick quiz.
Run these examples with local Python
A subprocess is another operating-system process. The browser Python editor cannot launch one, so the code cards on this page offer copying for a local Python 3.11 or later installation. Save an example in a Python file and run it with your installed interpreter. The examples launch that same interpreter with a small program; they do not require shell utilities, a network connection or an external service.
If Python is not installed yet, follow Python Fundamentals – Getting Started. Each checked panel describes an actual local run, not a browser simulation.
Start with an argument list and inspect the result
subprocess.run() starts a program, waits for it to finish and returns a CompletedProcess result. The first list item identifies the executable; each following item is one argument. sys.executable selects the current Python interpreter, and its -c option runs the source text in the next argument.
import subprocess
import sys
# Run a tiny child with the same interpreter as this parent program.
result = subprocess.run(
[sys.executable, "-c", "print('ready')"],
capture_output=True,
text=True,
encoding="utf-8",
check=True,
timeout=10,
)
print(result.returncode)
print(result.stdout.strip())
print(result.stderr == "")
# ✅ Expected output:
# 0
# ready
# TrueA zero return code conventionally means success. capture_output stores stdout and stderr in the result instead of letting the child write them directly to your terminal. text and encoding make those captured values strings. The final strip removes the trailing newline for this display; do not strip a protocol where leading or trailing whitespace is meaningful.
check=True asks run to raise an exception for a nonzero exit code. timeout bounds the wait for completion; the exact timing of process creation is platform-dependent. These are separate controls: neither replaces the other.
Spaces stay inside one argument; shell symbols stay data
Build a list instead of assembling a shell command string. A filename or label containing spaces remains one argument without adding quote characters yourself. With the default shell=False, a semicolon or a pipe in an argument is passed to the program, not interpreted as another command.
import subprocess
import sys
child = "import sys; print(len(sys.argv) - 1); print(sys.argv[1]); print(sys.argv[2])"
result = subprocess.run(
[sys.executable, "-c", child, "two words", "tea; milk | coffee"],
capture_output=True, text=True, encoding="utf-8", check=True, timeout=10,
)
print(result.stdout, end="")
# ✅ Expected output:
# 2
# two words
# tea; milk | coffeeThe child receives two arguments, including all the spaces and punctuation. Do not wrap "two words" in additional quote characters when it is already a list item. Avoid shell=True for untrusted input; introducing a shell changes which characters have executable meaning. An argument list also does not make an untrusted executable safe or prevent a program from interpreting an argument as one of its own options.
The executable and the input contract still need to be chosen deliberately. These examples use a known Python executable and data consumed by a tiny child, not a command assembled from user input.
Nonzero exit is a result until you ask for an exception
run defaults to check=False. A child can fail while the parent continues normally, so ignoring returncode is a real bug in automation. stdout and stderr are independent streams: error text on stderr does not by itself prove failure, and an empty stderr does not prove success.
import subprocess
import sys
child = "import sys; print('partial'); print('invalid row', file=sys.stderr); sys.exit(2)"
result = subprocess.run(
[sys.executable, "-c", child],
capture_output=True, text=True, encoding="utf-8", timeout=10,
)
print("status:", result.returncode)
print("stdout:", result.stdout.strip())
print("stderr:", result.stderr.strip())
# ✅ Expected output:
# status: 2
# stdout: partial
# stderr: invalid rowUse check=True when a later step requires successful completion. Catch CalledProcessError if this layer can report or recover from the failure. The exception retains the return code and captured streams; printing just those fields avoids dumping the command or sensitive arguments.
import subprocess
import sys
child = "import sys; print('invalid row', file=sys.stderr); sys.exit(2)"
try:
subprocess.run(
[sys.executable, "-c", child],
capture_output=True, text=True, encoding="utf-8", check=True, timeout=10,
)
except subprocess.CalledProcessError as error:
print(type(error).__name__)
print("failed:", error.returncode)
print(error.stderr.strip())
# ✅ Expected output:
# CalledProcessError
# failed: 2
# invalid rowThe catch reports failure; it does not turn the job into a success. Automation & Scripting for DevOps and System Tasks owns retry policy, idempotency and the status a scheduler should receive. Exception Handling owns exception propagation.
Choose bytes or text before sending input
Without text=True or an encoding, captured output is bytes. Specify an encoding when a program communicates through text, and make the child use the same encoding. The input option supplies stdin and closes it after the supplied data has been sent; the child does not have to read from your terminal.
import subprocess
import sys
result = subprocess.run(
[sys.executable, "-c", "import sys; sys.stdout.buffer.write(b'OK')"],
capture_output=True, check=True, timeout=10,
)
print(type(result.stdout).__name__)
print(result.stdout.decode("utf-8"))
# ✅ Expected output:
# bytes
# OKimport subprocess
import sys
child = "import sys; data = sys.stdin.buffer.read().decode('utf-8'); sys.stdout.buffer.write(data.upper().encode('utf-8'))"
result = subprocess.run(
[sys.executable, "-c", child],
input="tea and milk", capture_output=True,
text=True, encoding="utf-8", check=True, timeout=10,
)
print(result.stdout)
# ✅ Expected output:
# TEA AND MILKIn text mode, input is a string. In binary mode, it must be bytes. The child above explicitly decodes and encodes UTF-8, so it does not rely on the local terminal encoding. A real worker protocol should also define message framing, maximum size and how failures are represented.
Integrating Python with Other Languages (C, Rust & More) owns the choice between a process protocol and native bindings. This lesson owns starting and observing the process.
Do not ask for two incompatible capture configurations
capture_output=True already configures both stdout and stderr as pipes. Supplying stdout or stderr as well contradicts that shortcut and fails before the child starts.
import subprocess
import sys
subprocess.run(
[sys.executable, "-c", "print('ready')"],
capture_output=True,
stdout=subprocess.PIPE,
)
# ✅ Expected error:
# ValueError: stdout and stderr arguments may not be used with capture_output.Use capture_output by itself for two separate streams. To merge stderr into stdout instead, omit capture_output and set stdout=PIPE with stderr=STDOUT. Merging loses the distinction between the two streams, so it is unsuitable when stdout must contain only machine-readable data.
Bound execution and distinguish failure to start
A nonzero exit means the child started and reported a failure. A missing executable can raise FileNotFoundError before there is any child result. A timeout raises TimeoutExpired; run kills and waits for its child before raising, but does not promise to manage every descendant process a child may start. Report these as different failure categories.
All successful launch examples use a ten-second timeout as a safety bound for tiny local programs. They do not intentionally sleep or print elapsed time, because a timing-dependent demonstration would make the expected panel unreliable. In a real workflow, select a deadline appropriate to the command and handle TimeoutExpired at the layer that can decide whether retrying is safe.
Captured output is held in memory. For large output, direct a stream to a suitable file or design incremental processing with Popen. Popen gives lower-level control, but reading one pipe while ignoring another can deadlock if the unread pipe fills; communicate() is the normal way to exchange bounded input and output without that mistake. The subprocess reference documents those lifecycle details.
Complete a checked child invocation
Fill the three options that capture output, decode it as text and reject a failing exit status. Copy the completed example into local Python; its child prints one fixed message.
import subprocess
import sys
result = subprocess.run(
[sys.executable, "-c", "print('checked')"],
___=True,
# 👉 replace ___ with capture_output
___=True,
# 👉 replace ___ with text
___=True,
# 👉 replace ___ with check
encoding="utf-8", timeout=10,
)
print(result.stdout.strip())
# ✅ Expected output:
# checkedReport three independent worker outcomes
Run three synthetic Python workers named clean, invalid and done, with exit codes 0, 2 and 0. Each worker prints its own name, then exits with its assigned code. Capture output, use a timeout, and print each name, return code and success boolean. Finally print how many workers failed. These are independent test workers, so continuing after invalid is deliberate; they do not run actual build or deployment tools.
import subprocess
import sys
failures = 0
for name, code in [("clean", 0), ("invalid", 2), ("done", 0)]:
child = "import sys; print(sys.argv[1]); sys.exit(int(sys.argv[2]))"
result = subprocess.run(
[sys.executable, "-c", child, name, str(code)],
capture_output=True, text=True, encoding="utf-8", timeout=10,
)
success = result.returncode == 0
failures += not success
print(result.stdout.strip(), result.returncode, success)
print("failures:", failures)
# ✅ Expected output:
# clean 0 True
# invalid 2 False
# done 0 True
# failures: 1A dependent pipeline would stop before a later worker consumed invalid output. Here a launch error or timeout still propagates instead of being counted as an ordinary nonzero exit. A production caller can catch and report those separately, while preserving the overall failure.
Practice quiz
What does this program print? import subprocess import sys r = subprocess.run([sys.executable, "-c", "pass"], capture_output=True, timeout=10) print(r.returncode)
- 1
- 0
- None
- True
Answer: 0. A child that finishes normally has a zero return code.
What does this program print? import subprocess import sys r = subprocess.run([sys.executable, "-c", "import sys; sys.exit(3)"], capture_output=True, timeout=10) print(r.returncode)
- 0
- None
- CalledProcessError
- 3
Answer: 3. check defaults to False, so the nonzero code remains a result to inspect.
What does this program print? import subprocess import sys r = subprocess.run([sys.executable, "-c", "print('ok')"], capture_output=True, timeout=10) print(type(r.stdout).__name__)
- bytes
- str
- list
- NoneType
Answer: bytes. Captured streams are bytes unless text mode or an encoding is selected.
What does this program print? import subprocess import sys r = subprocess.run([sys.executable, "-c", "print('ok')"], capture_output=True, text=True, encoding="utf-8", timeout=10) print(r.stdout.strip())
- b'ok'
- 0
- ok
- None
Answer: ok. Text mode decodes the captured stream; strip removes its newline.
What does check=True add to subprocess.run()?
- It captures stdout automatically.
- It raises CalledProcessError for a nonzero exit code.
- It installs a missing executable.
- It makes every command safe.
Answer: It raises CalledProcessError for a nonzero exit code.. Checking the exit code is independent of capture, executable selection and input validation.
How should a two-word label be passed to a known executable?
- Split it into two list elements.
- Add a shell pipe.
- Turn on shell=True.
- Use one list element containing both words.
Answer: Use one list element containing both words.. Each list element is one argument; no shell quoting is needed for the spaces.
Which failure means a launched child exceeded its configured wait limit?
- TimeoutExpired.
- A zero return code.
- FileNotFoundError for the executable.
- An empty stderr string.
Answer: TimeoutExpired.. TimeoutExpired is distinct from an unsuccessful exit or a failure to start.
Why should a JSON worker keep diagnostic logs out of stdout?
- stderr is always hidden.
- stdout cannot contain Unicode.
- Extra log text can invalidate the JSON response.
- check=True removes log lines.
Answer: Extra log text can invalidate the JSON response.. A machine-readable stream needs a defined protocol; separate diagnostic output from the response.
Continue this course
- Previous: Randomness with the random Module
- Next: Hashing with hashlib — Compute SHA-256/HMAC digests, hash files, and understand salts vs encryption
- Quick reference: Python cheat sheet