Retries & backoff
What to retry, what to fix, and how to back off.
Kontinent handles provider-level failover for you, but your client still needs a sane retry strategy for the errors that reach it.
Decide by status code
Retriable
429 rate_limit_exceeded, 429 upstream_rate_limited, 500, 502 upstream_error, 503 no_provider_available, 503 transcriptions_busy. Use exponential backoff with jitter.
Not retriable as-is
400, 401, 403, 404, 429 spend_limit_exceeded. Fix the request, the key or the limit first.
401 invalid_api_key: the key is wrong or revoked. Retrying with the same key will not help.402 insufficient_credits: retriable only after a top-up lands.403 segment_not_permitted: choose a model from another provider instead of retrying.429 rate_limit_exceeded: your key's RPM window is full. Back off for at least the 60 s window.429 upstream_rate_limited: the provider throttled us, not you. Retry with backoff.429 spend_limit_exceeded: a spend limit is used up. Retrying does not help until the window resets or the limit is raised.
See the full error taxonomy for every code.
Streaming needs a different rule
Failover between providers only happens before the first byte of a stream. Once tokens start arriving, an upstream failure arrives as a final error chunk (finish_reason: "error", no data: [DONE] after it), not as an HTTP error. Treat a stream that ends without data: [DONE] as a retriable failure on your side, and remember that the tokens generated up to that point are already billed. See Streaming.
A reasonable default
If a response carries Retry-After, wait at least that long: 60 seconds on 429 rate_limit_exceeded. Otherwise, exponential backoff with jitter, starting around 200 ms to 500 ms and capping at a few seconds, handles 429 upstream_rate_limited and transient 5xx errors well. Stop retrying after a small, fixed number of attempts and surface the failure; do not retry indefinitely.