D-07 · API · RATE LIMITS
API rate limits and 429 retry strategy
A launch-hour retry storm turns one throttled minute into a thirty-minute outage. This scenario designs queues, backoff, and budgets that absorb 429s gracefully.
Symptom
A product launch fires ten thousand welcome emails in three minutes. The API returns 429s, every worker retries immediately, and the retry wave itself sustains the throttle. Latency climbs, timeouts cascade into duplicate sends when some first attempts actually succeeded, and users receive two or three welcomes. The dashboard shows sends “recovering” while support handles duplicates.
The secondary symptom is key exhaustion: one noisy bulk job consumes the shared API budget, starving transactional sends that share the key and quota.
Cause
Rate limits protect shared infrastructure per key, per account, and per endpoint — often with burst and sustained dimensions. Naive clients treat 429 as “try again now,” synchronizing retries into thundering herds. Missing Retry-After handling, absent jitter, and unbounded concurrency convert a routine throttle into a self-inflicted outage. Shared keys without per-workload budgets let bulk workloads cannibalize transactional latency SLAs.
Duplicate sends arise because timeouts are ambiguous: the server may have accepted and queued the message while the client saw a timeout. Retrying without idempotency creates a second message, not a resumed one.
Fix
Front the API with a durable queue per message class — transactional, lifecycle, bulk — each with independent concurrency caps and rate budgets. On 429 or 5xx, apply exponential backoff with full jitter (for example, base 1s doubling to a 5-minute cap, randomized), honor Retry-After headers when present, and bound total attempts with dead-lettering after exhaustion. Separate API keys or subaccounts per class so bulk throttles never block password resets.
Attach idempotency keys to every send so retries are safe replays rather than new messages, and make workers crash-safe: dequeue, attempt with key, record provider message ID, acknowledge. Replay the dead-letter queue manually after incidents, deduplicating by key.
Prevention
Load-test at 3× launch volume against sandbox limits, chart per-key consumption with alerts at 70% of quota, and schedule bulk outside transactional peaks. Document each vendor’s burst vs sustained limits in a runbook (they differ), and review retry metrics — not just send counts — in weekly reliability reviews. A healthy system’s retry rate stays low and jittered; a spiky one predicts the next outage.
Worked example
A launch fires 10,000 welcomes in three minutes against a burst-capped API. The first hundred 429s trigger the new behavior: workers back off with jittered exponential delays, honor Retry-After, and shed bulk-concurrency while the transactional queue keeps its own budget untouched. Sends stretch to twenty minutes instead of three — and every message delivers exactly once.
The previous launch without this design completed “faster” on paper while generating 300 duplicates and starving password resets for half an hour. The comparison ends the internal debate about whether pacing “slows us down”: measured by delivered inboxes and zero support tickets, the slower schedule is dramatically faster.