Timeouts: junior vs mid-level vs senior

Every service that calls another service has to deal with timeouts. How a developer handles them says a lot about their level:

• Junior: misses adding timeouts.
• Mid-level: adds timeouts.
• Senior: adds timeouts and designs for the unknown.

Let’s walk through all three with one example: your order service calls a payment service owned by another team.

Junior: no timeout

Many HTTP clients wait forever by default. Python’s requests is the classic example. Its docs say it plainly: if you don’t set a timeout, requests do not time out, and “nearly all production code should use this parameter in nearly all requests.” Go’s http.Client is the same: its default timeout is zero, which means no timeout.

So this innocent-looking line can wait forever:

resp = requests.post(PAYMENT_URL, json=payload)   # no timeout: waits as long as payment takes

Everything works in testing. Then one day the payment service gets slow, say 30 seconds per call.

Each request your service handles now holds a worker thread for 30 seconds. Say your service has 200 threads and gets 50 requests a second:

threads needed = requests per second × wait time
               = 50 × 30 = 1,500   (you have 200)

time to run out = 200 ÷ 50 = 4 seconds

In about 4 seconds every thread is stuck waiting on payment. Now even requests that never touch payment, like your product pages or /health, can’t get a thread. Your service is down.

Payment was only slow. Your service is down, because of someone else’s slowness. That’s a cascading failure, and a missing timeout is one of the most common ways it starts.

Mid-level: add a timeout (and pick the value properly)

The fix is one argument. Set a timeout on every network call: HTTP, database, Redis, message queues.

resp = requests.post(PAYMENT_URL, json=payload, timeout=(3, 1))   # (connect, read) in seconds

The harder question is how much. Don’t guess. Look at the payment service’s tail latency, its P99: the time within which 99% of calls finish. (I explained P95 and P99 in an earlier reel; it’s on my Instagram, @suryatechhub.)

If payment’s P99 is 300 ms, a timeout of around 1 second leaves headroom for normal slow calls while still cutting off the stuck ones.

• Too low (say 200 ms): you fail calls that would have succeeded. If you also retry, you add load to a service that’s already slow (see Retries done wrong).
• Too high (say 30 s): you’re back to threads piling up.

Amazon’s Builders’ Library takes this further: pick the percentile that matches the false-timeout rate you can accept. They use p99.9 of the downstream latency, so only 0.1% of calls time out by mistake. P99 is a good starting point; tighten it with real data.

One detail with requests: the read timeout is the maximum wait between bytes, not for the whole response. A server that trickles bytes slowly can keep a call open longer than the number suggests. If you need a hard limit on total time, enforce a deadline around the call.

Now slow calls are cut after a second, threads are freed, and your service stays up. Most developers stop here.

Senior: a timeout doesn’t mean it failed

Here’s the catch. The payment call timed out after 2 seconds. Your code shows “Payment failed”. Correct?

No. A timeout only means you stopped waiting. It tells you nothing about what happened on the other side. The payment service may still be working on it, and it may charge the customer two or three seconds later.

So the customer’s card is charged, but your app says “failed”. They click Pay again, or your code retries, and they’re charged twice.

That’s why every network call has three outcomes, not two:

Outcome	What you know
✅ Success	It worked.
❌ Failure	It didn’t work (for example, a 400 or a declined card).
❓ Unknown	You don’t know. The request may have succeeded, failed, or never arrived.

A timeout is unknown. Treating it as failure is the bug.

A senior designs for the unknown. Two common ways:

1. Retry with an idempotency key

Send a unique key with the request, and send the same key on every retry of that payment. The payment service remembers keys it has already processed. If the first attempt went through, the retry gets back the original result instead of charging again. Stripe’s API works exactly this way with its Idempotency-Key header.

import random
import time

import requests


class PaymentPending(Exception):
    """The payment result is unknown: show "processing", check again later."""


def charge(order_id: str, amount: int) -> dict:
    key = f"order-{order_id}"   # the same key on every attempt for this order
    for attempt in range(3):
        try:
            resp = requests.post(
                PAYMENT_URL,
                json={"order_id": order_id, "amount": amount},
                headers={"Idempotency-Key": key},
                timeout=(3, 1),
            )
            resp.raise_for_status()
            return resp.json()
        except (requests.Timeout, requests.ConnectionError):
            if attempt < 2:
                time.sleep(random.uniform(0, 2 ** attempt))   # backoff + jitter before the next try
    raise PaymentPending(order_id)   # still unknown: don't call it "failed"

This only works if the payment service supports idempotency keys. I covered how to build one in Idempotency.

2. Check the status before you retry

If the service lets you look up a payment, ask before you act:

GET /payments/order-42   →  paid?     show success, don't retry
                         →  pending?  show "Payment processing", check again later
                         →  not found? safe to retry

And in the UI, show “Payment processing”, not “Payment failed”, while the result is unknown. For anything that stays unknown, a background job can reconcile later against the payment provider’s records.

Either way, you avoid false alarms about failures, and you never charge twice.

Checklist

• ✅ Set a timeout on every network call. Don’t trust client defaults.
• ✅ Pick the value from the dependency’s P99 (or P99.9), with headroom.
• ✅ Treat a timeout as unknown, not failure.
• ✅ Retry only with an idempotency key, or check the status first.
• ✅ Show “processing” while the result is unknown, and reconcile later.

A timeout protects your service. It doesn’t tell you what happened. Design for the unknown.