Idempotency: why a retry should never charge twice

You tap Pay ₹500. The spinner keeps spinning, and the app shows “Something went wrong”. So the app retries, or you tap again.

Did the first payment go through? Maybe. The request may have reached the server, charged your card, and only the response got lost on the way back. If the server treats the retry as a brand-new payment, you’ve just paid ₹1,000.

This isn’t a rare edge case. Timeouts, dropped mobile connections and load balancer resets happen all day in production. Any system that retries, and every real system does, needs to make retries safe. That property has a name.

What idempotency means

An operation is idempotent if doing it once or many times has the same effect.

Setting a user’s email to a@b.com is idempotent: do it five times and the email is still a@b.com. Adding ₹500 to a balance is not: do it five times and you’ve added ₹2,500.

The HTTP specification (RFC 9110) already defines which methods should behave this way:

Method	Idempotent?	Why
GET, HEAD, OPTIONS	Yes	They only read
PUT	Yes	“Set this resource to exactly this” gives the same result every time
DELETE	Yes	Once it’s deleted, it stays deleted
POST	No	“Create a new thing” creates another one each time
PATCH	Not guaranteed	Depends on the change, like “increment by 1”

Payments, orders and bookings are almost always POST. That’s exactly why they need extra help: an idempotency key.

The bug: a payment endpoint that charges on every retry

Here’s a simple FastAPI endpoint. It looks fine, and it will double-charge people.

import uuid

from fastapi import FastAPI
from pydantic import BaseModel

app = FastAPI()


class PaymentRequest(BaseModel):
    user_id: str
    amount: int  # in paise: ₹500 = 50000


def charge_card(user_id: str, amount: int) -> str:
    """Stand-in for a real payment gateway call. Returns a payment id."""
    return f"pay_{uuid.uuid4().hex[:8]}"


@app.post("/payments")
def create_payment(req: PaymentRequest):
    payment_id = charge_card(req.user_id, req.amount)
    return {"payment_id": payment_id, "status": "charged"}

Every request, including every retry, calls charge_card again. The server has no way to tell “a new payment” from “the same payment, asked again”.

The fix: an idempotency key

The idea takes three steps:

1. The client creates a unique key for each payment attempt, such as a UUID, and sends it in an Idempotency-Key header.
2. The server saves the key with the result the first time it processes the request.
3. If the same key arrives again, the server returns the saved result instead of doing the work again.

Here’s the endpoint with idempotency added. It uses an in-memory dictionary so the logic is easy to follow:

import hashlib
import threading
from typing import Annotated

from fastapi import FastAPI, Header, HTTPException

app = FastAPI()

_lock = threading.Lock()
_store: dict[str, dict] = {}  # key -> {"fingerprint", "status", "response"}


def fingerprint(req: PaymentRequest) -> str:
    """A hash of the request body, to catch a key reused for a different payment."""
    return hashlib.sha256(req.model_dump_json().encode()).hexdigest()


@app.post("/payments")
def create_payment(
    req: PaymentRequest,
    idempotency_key: Annotated[str | None, Header()] = None,
):
    if idempotency_key is None:
        raise HTTPException(400, "Idempotency-Key header is required")

    store_key = f"{req.user_id}:{idempotency_key}"  # keys are scoped per user
    fp = fingerprint(req)

    # 1. Claim the key, or find out what already happened with it.
    with _lock:
        record = _store.get(store_key)
        if record is None:
            _store[store_key] = {"fingerprint": fp, "status": "in_progress", "response": None}
        elif record["fingerprint"] != fp:
            raise HTTPException(422, "Idempotency-Key was reused for a different request")
        elif record["status"] == "in_progress":
            raise HTTPException(409, "A request with this Idempotency-Key is still in progress")
        else:
            return record["response"]  # 3. Same key again: replay the saved result.

    # 2. First time we've seen this key: do the real work.
    try:
        payment_id = charge_card(req.user_id, req.amount)
    except Exception:
        with _lock:
            del _store[store_key]  # nothing was charged, so the client may retry
        raise

    response = {"payment_id": payment_id, "status": "charged"}
    with _lock:
        _store[store_key].update(status="done", response=response)
    return response

(PaymentRequest and charge_card are the same as in the first example.)

A few details here are what separate a demo from a design:

• in_progress status. If a retry arrives while the first request is still charging the card, we reply 409 Conflict instead of charging again.
• Fingerprint check. If a buggy client reuses the same key for a different payment, we reject it with 422 instead of silently returning the wrong result.
• Keys scoped per user. Two users who happen to send the same key never collide.
• The lock. FastAPI runs normal def endpoints in a thread pool, so two requests really can arrive at the same moment. The check-and-claim step must be atomic.

These status codes follow the IETF’s draft standard for the Idempotency-Key header, which many public APIs already use.

Production version: Redis instead of a dictionary

The dictionary above only works inside one process. The moment you run two instances behind a load balancer, a retry can land on the other instance and miss the saved key. In production, the store must be shared and must expire old keys. Redis is a common choice, because SET ... NX claims a key atomically across every instance:

import json
from typing import Annotated

import redis.asyncio as redis
from fastapi import FastAPI, Header, HTTPException

app = FastAPI()
r = redis.Redis(host="localhost", port=6379)
KEY_TTL_SECONDS = 24 * 60 * 60  # keep keys for 24 hours


@app.post("/payments")
async def create_payment(
    req: PaymentRequest,
    idempotency_key: Annotated[str | None, Header()] = None,
):
    if idempotency_key is None:
        raise HTTPException(400, "Idempotency-Key header is required")

    store_key = f"idem:{req.user_id}:{idempotency_key}"
    fp = fingerprint(req)

    # SET NX succeeds for exactly one request, across all app instances.
    claimed = await r.set(
        store_key,
        json.dumps({"fingerprint": fp, "status": "in_progress"}),
        nx=True,
        ex=KEY_TTL_SECONDS,
    )
    if not claimed:
        raw = await r.get(store_key)
        if raw is None:  # expired between the two calls; ask the client to retry
            raise HTTPException(409, "Please retry")
        record = json.loads(raw)
        if record["fingerprint"] != fp:
            raise HTTPException(422, "Idempotency-Key was reused for a different request")
        if record["status"] == "in_progress":
            raise HTTPException(409, "A request with this Idempotency-Key is still in progress")
        return record["response"]

    try:
        payment_id = await charge_card_async(req.user_id, req.amount)  # your async gateway client
    except Exception:
        await r.delete(store_key)
        raise

    response = {"payment_id": payment_id, "status": "charged"}
    await r.set(
        store_key,
        json.dumps({"fingerprint": fp, "status": "done", "response": response}),
        ex=KEY_TTL_SECONDS,
    )
    return response

fingerprint and PaymentRequest are the same as before. A database table with a unique constraint on (user_id, idempotency_key) works just as well, and is often better for payments, because you can save the key in the same transaction as the payment record.

The client’s side of the deal

Idempotency only works if the client plays along. The rule is simple: create the key once per payment, and reuse it on every retry.

import time
import uuid

import httpx


def pay(user_id: str, amount: int) -> dict:
    key = str(uuid.uuid4())  # one key for this payment, created once

    for attempt in range(3):
        try:
            resp = httpx.post(
                "https://api.example.com/payments",
                json={"user_id": user_id, "amount": amount},
                headers={"Idempotency-Key": key},  # the same key on every retry
                timeout=5,
            )
            if resp.status_code == 409:  # the first attempt is still running
                time.sleep(2**attempt)
                continue
            resp.raise_for_status()
            return resp.json()
        except httpx.TransportError:  # timeout, connection reset, and so on
            time.sleep(2**attempt)

    raise RuntimeError("Payment failed after 3 attempts")

If the client generated a new key on each retry, the server would see three different payments, and you’d be back to double charges.

One trap to know about

Look at the except block on the server: if charge_card raises, we delete the key so the client can retry. But what if the gateway did charge the card and only the response to us timed out? Deleting the key would allow a second charge.

The real fix is to make your call to the gateway idempotent too. Many payment gateways, Stripe for example, accept their own idempotency key. Pass one through (it can be derived from your key), and the whole chain becomes safe end to end. This is the kind of detail that shows up in senior-level design reviews: idempotency only protects the steps it actually covers.

Checklist

• ✅ The client creates the key (a UUID) and reuses it on retries.
• ✅ The server claims the key atomically before doing any work.
• ✅ The server stores the response and replays it for repeat requests.
• ✅ Keys are scoped per user and expire (for example, after 24 hours).
• ✅ The same key with a different body is rejected (422), and a key still in progress returns 409.
• ✅ Calls to downstream services (like the payment gateway) are idempotent too.

Networks fail. Retries happen. Idempotency makes retries safe.