REST API Clients

Published by Brayan K

Master building robust API clients, handling authentication, retries, pagination, webhooks, and async operations for production-grade integrations

Part of the free Python course at LearnCodingFast — hands-on lessons with examples you run in your browser, plus practice exercises and a quick quiz.

A successful response can still be unusable

An external service can return valid JSON with the wrong shape, an expired credential, or only the first page of a result. A reusable client handles these boundaries in one place so the rest of your program receives useful values or a clear failure. It should not turn every failure into an empty list: callers would mistake an outage for “no records”.

The HTTP methods, request parameters and basic status handling belong to HTTP Requests & APIs. Parsing and writing JSON belong to Working with JSON; here we validate the service's contract after decoding it.

Suppose the service promises a JSON object with a nonnegative integer remaining. A numeric-looking string is not that integer. The following response is deliberately a local fixture, not a request to a live service.

import json

def remaining_from(body):
    data = json.loads(body)
    if not isinstance(data, dict):
        raise ValueError("response must be an object")
    remaining = data.get("remaining")
    if type(remaining) is not int or remaining < 0:
        raise ValueError("remaining must be a nonnegative integer")
    return remaining

remaining_from('{"remaining": "2"}')

# ✅ Expected error:
# ValueError: remaining must be a nonnegative integer

The fix belongs at the contract boundary: correct the producer or deliberately support a documented alternate format. Blindly calling int() can hide an upstream change. Exact type(... ) is int is intentional here because JSON booleans become Python booleans, which otherwise satisfy isinstance(value, int).

import json

def remaining_from(body):
    data = json.loads(body)
    if not isinstance(data, dict):
        raise ValueError("response must be an object")
    value = data.get("remaining")
    if type(value) is not int or value < 0:
        raise ValueError("remaining must be a nonnegative integer")
    return value

for body in ['{}', '{"remaining": true}', '{"remaining": 0}']:
    try:
        print("accepted:", remaining_from(body))
    except ValueError as error:
        print("rejected:", error)


# ✅ Expected output:
# rejected: remaining must be a nonnegative integer
# rejected: remaining must be a nonnegative integer
# accepted: 0

There are three separate checks: transport succeeded, HTTP status matches the operation, and payload matches its schema. JSON decoding alone proves none of the others. A documented 204 response has no body to decode; a 200 response can still contain invalid business data.

Keep credentials and transport behind one client

This client accepts a transport: a function that performs a request and returns a status and response body. The runnable example passes a recording fixture, so no socket is opened and no credential leaves the program. In an application, an adapter would use a configured HTTPS base URL and a reusable requests Session, converting its response into this small contract.

import json

class CatalogClient:
    def __init__(self, transport, token):
        if not token:
            raise ValueError("a token is required")
        self.transport = transport
        self.token = token

    def name_for(self, item_id):
        if type(item_id) is not int or item_id < 1:
            raise ValueError("item_id must be a positive integer")
        status, body = self.transport(
            method="GET", resource=f"/items/{item_id}",
            headers={"Authorization": "Bearer " + self.token},
            timeout=(2, 5),
        )
        if status == 404:
            return None
        if status != 200:
            raise RuntimeError("catalog request failed")
        data = json.loads(body)
        if not isinstance(data, dict) or not isinstance(data.get("name"), str):
            raise ValueError("item name must be text")
        return data["name"]

calls = []
def fixture_transport(**request):
    calls.append(request)
    return 200, '{"name": "Notebook"}'

client = CatalogClient(fixture_transport, "fixture-token-not-a-secret")
print(client.name_for(7))
print("method:", calls[0]["method"])
print("timeouts:", calls[0]["timeout"])
print("authenticated:", "Authorization" in calls[0]["headers"])


# ✅ Expected output:
# Notebook
# method: GET
# timeouts: (2, 5)
# authenticated: True

The client chooses the endpoint shape and validates its identifier. The transport owns network I/O; the client owns response interpretation. Returning None for 404 is an explicit choice for this lookup, not a rule for every endpoint. Authentication failures should reach the caller as failures, with secrets and full response bodies excluded from logs.

Load a real token from your environment or secret store at application startup and pass it in. Keep it out of browser code, URLs, source control and exception messages. A requests timeout pair limits connection establishment and socket read inactivity; it is not an overall deadline for a download plus every retry. A production operation also needs a total time budget and response-size limits.

An asynchronous client still needs these contracts and a bound on requests in flight; Advanced Async & Await Patterns owns asynchronous coordination. A webhook reverses the direction: another service calls your server endpoint, whose request handling belongs to Building a REST API; a real webhook also needs the provider's signature and replay-protection rules.

Retry a read only while the policy permits it

A timeout can happen after the server accepted a write. Repeating “charge this card” might charge twice. This demonstration retries a read-only GET on 429 or 503, with at most three attempts. It never retries authentication errors, malformed payloads or an arbitrary POST. A write needs the service's documented idempotency mechanism before automatic replay is safe.

For a 429, the fixture supplies Retry-After as whole seconds. If the requested delay exceeds our remaining waiting budget, we stop instead of retrying earlier than the server asked. The injected wait records delays without sleeping, keeping the exercise repeatable.

def read_with_retry(send, wait, max_attempts=3, delay_budget=5):
    if max_attempts < 1:
        raise ValueError("max_attempts must be positive")
    waited = 0
    for attempt in range(max_attempts):
        status, retry_after, body = send()
        if status == 200:
            return body
        if status not in (429, 503):
            raise RuntimeError("non-retryable response")
        if attempt == max_attempts - 1:
            raise RuntimeError("attempt limit reached")
        delay = 2 ** attempt
        if retry_after is not None:
            if not retry_after.isdecimal():
                raise ValueError("fixture requires Retry-After seconds")
            delay = int(retry_after)
        if waited + delay > delay_budget:
            raise RuntimeError("retry delay exceeds budget")
        wait(delay)
        waited += delay

responses = iter([(503, None, "busy"), (429, "3", "slow down"), (200, None, "ready")])
delays = []
attempts = []
def fixture_send():
    attempts.append("GET")
    return next(responses)

print(read_with_retry(fixture_send, delays.append))
print("attempts:", len(attempts))
print("requested waits:", delays)


# ✅ Expected output:
# ready
# attempts: 3
# requested waits: [1, 3]

The budget above covers requested delays only. It does not measure request duration. A real adapter must also handle connection failures, documented retryable statuses and an HTTP-date form of Retry-After. Use a maintained retry implementation configured for that service, add jitter to spread real clients' retries, and enforce a total deadline. Jitter and wall-clock dates are deliberately absent from this deterministic fixture.

Do not treat every 5xx as transient: for example, “not implemented” is not likely to change by waiting. Refreshing a credential should also be bounded, rather than looping between 401 and refresh forever. HTTP's retry semantics are documented in RFC 9110, idempotent methods.

Pagination needs an end condition and a limit

A page containing two records does not prove the collection contains only two. Follow the service's continuation token until it says there is no next page. Keep the cursor opaque, detect repeats, and cap the number of requests so a faulty service cannot send your client around a cycle forever.

def collect_names(read_page, max_pages=5):
    cursor = None
    seen = set()
    names = []
    for _ in range(max_pages):
        if cursor in seen:
            raise ValueError("pagination cursor repeated")
        seen.add(cursor)
        page = read_page(cursor)
        if not isinstance(page, dict) or not isinstance(page.get("items"), list):
            raise ValueError("page must contain an items list")
        for item in page["items"]:
            if not isinstance(item, dict) or not isinstance(item.get("name"), str):
                raise ValueError("item name must be text")
            names.append(item["name"])
        cursor = page.get("next")
        if cursor is None:
            return names
        if not isinstance(cursor, str):
            raise ValueError("next cursor must be text or null")
    raise RuntimeError("page limit reached")

pages = {
    None: {"items": [{"name": "Ada"}], "next": "cursor-b"},
    "cursor-b": {"items": [{"name": "Bo"}], "next": None},
}
print(collect_names(pages.__getitem__))


# ✅ Expected output:
# ['Ada', 'Bo']

This fixture contract treats a missing next as the end. If the real API requires the field, reject a missing field instead. Some services return a full continuation URL; verify its allowed HTTPS origin before attaching credentials. Never follow an arbitrary URL with the same Authorization header merely because it appeared in a response.

seen = set()
cursor = None
next_cursors = ["cursor-b", "cursor-b", None]

for next_cursor in next_cursors:
    if cursor in seen:
        raise ValueError("pagination cursor repeated")
    seen.add(cursor)
    cursor = next_cursor

# ✅ Expected error:
# ValueError: pagination cursor repeated

The loop stops because it would request the same page again. Increasing the page limit would only hide that bug. For large collections, process validated pages incrementally rather than collecting everything; that storage decision belongs to Working with Files, Streams & Large Datasets.

Complete the retry decision without retrying a write

Fill the method, allowed-status tuple and attempt bound. The attempt number here counts the request that just failed. There are three total attempts, not three retries after the first request.

def should_retry(method, status, attempt):
    return method == ___ and status in ___ and attempt < ___

for case in [("GET", 503, 1), ("POST", 503, 1), ("GET", 401, 1), ("GET", 503, 3)]:
    print(should_retry(*case))

# 👉 replace ___ with "GET"
# 👉 replace ___ with (429, 503)
# 👉 replace ___ with 3


# ✅ Expected output:
# True
# False
# False
# False

Translate a lookup response without hiding outages

Write lookup(status, body). For 404, return None. For 200, decode an object containing a text name and return that name. For any other status, raise RuntimeError("lookup failed") without including the response body. Reject an invalid name with ValueError("name must be text"). Exercise it with the four fixture responses in the solution; display the returned value or the error's type and message. No network call is needed.

import json

def lookup(status, body):
    if status == 404:
        return None
    if status != 200:
        raise RuntimeError("lookup failed")
    data = json.loads(body)
    if not isinstance(data, dict) or not isinstance(data.get("name"), str):
        raise ValueError("name must be text")
    return data["name"]

fixtures = [(200, '{"name": "Ada"}'), (404, ""), (503, "private diagnostic"), (200, '{"name": 7}')]
for status, body in fixtures:
    try:
        print(lookup(status, body))
    except (ValueError, RuntimeError) as error:
        print(type(error).__name__ + ":", error)


# ✅ Expected output:
# Ada
# None
# RuntimeError: lookup failed
# ValueError: name must be text

The fixture proves the client's decisions, not the remote service's current schema or availability. Test the real adapter separately against the provider's sandbox, without live credentials in fixtures. Requests' timeout and response documentation explains the network behaviour the adapter must preserve.

Practice quiz

An API field must be an integer, excluding booleans. What does print(type(True) is int) print?

  • True
  • TypeError
  • False
  • 1

Answer: False. True has type bool, so the exact-type check rejects it. isinstance(True, int) would accept it; a successful HTTP status does not validate the field.

A payment POST times out after sending. The client cannot tell whether the charge happened. What must it establish before automatically replaying the request?

  • Whether the provider guarantees a safe replay
  • Whether a longer read timeout can be configured
  • Whether a new request identifier can be generated
  • Whether the failed response body can be ignored

Answer: Whether the provider guarantees a safe replay. A timeout does not prove the server did nothing. Follow the provider's idempotency contract, including reuse of the same key when required; neither more time nor a new identifier prevents duplicate effects.

Retry-After requires a 30-second wait, but this operation has only 5 seconds left in its allowed wait budget. What should this retry policy do?

  • Retry after 5 seconds to stay within the budget
  • Retry immediately because the budget is nearly spent
  • Wait 30 seconds and extend the budget for this retry
  • Stop this operation without sending another attempt

Answer: Stop this operation without sending another attempt. The client cannot both honor that server delay and stay within its remaining budget. End with a bounded failure instead of retrying too early or silently extending the limit.

The contract uses None only for a missing record. The service keeps returning 503 until retries are exhausted. Which result preserves that distinction?

  • Return None to mark the service as unavailable
  • Raise the service-failure exception for the caller
  • Return an empty list as a successful empty result
  • Return the last body without validating its schema

Answer: Raise the service-failure exception for the caller. A service failure is not evidence that the record is absent. Raise a failure the caller can handle; None and empty success values would hide the outage.

A pagination response repeats a cursor that the client has already requested. Which change prevents an unbounded loop?

  • Discard duplicate records and keep following cursors
  • Reset to the first page whenever a cursor repeats
  • Reject a cursor already present in the seen set
  • Request fewer records each time the cursor repeats

Answer: Reject a cursor already present in the seen set. Track requested cursors and fail on repetition. Removing duplicate records does not stop requests; resetting the cursor or shrinking the page size does not establish progress.

Each request already has connect and read-inactivity timeouts, but a sequence of retries runs too long overall. Which missing limit addresses the whole operation?

  • A total deadline shared by requests and retry waits
  • A separate read timeout restarted for every request
  • A separate connect timeout restarted for every request
  • A fresh wait budget whenever the server sends a response

Answer: A total deadline shared by requests and retry waits. Per-request or inactivity limits do not cap the whole retry sequence. The lesson's wait-budget fixture bounds delays only; production code also needs a deadline covering requests.

A response supplies an absolute continuation URL on an unfamiliar host. The client normally sends a bearer token. What must happen before the next request?

  • Send the token if the continuation URL uses HTTPS
  • Trust the host because the URL came from JSON
  • Follow the URL and validate its response afterward
  • Check its origin against the credential allow-list

Answer: Check its origin against the credential allow-list. HTTPS alone does not make a host an authorized recipient of the token. Validate the continuation origin before sending credentials; do not forward them blindly.

A recording transport test has checked the requested path, fixture status handling and fixture schema validation. Which claim still needs separate evidence?

  • The test sent the expected path to its fake transport
  • The real service accepts the client's request contract
  • The fixture error status followed the tested failure path
  • The fixture payload passed the tested schema validation

Answer: The real service accepts the client's request contract. A recording fake can prove how this client handles the supplied fixtures. It cannot prove that a real service, credential or network adapter behaves the same way.

Continue this course

Related lessons