Failover & retries
What happens when a backend fails, and who pays for the attempt.
When a backend cannot serve your request, the next one gets it — inside the same request, before any answer reaches you. You do not retry, and you do not see the attempt that failed. It is in the usage log afterwards if you want it.
This is what makes the GPU network usable in production: the network dropping a request is absorbed here, not passed on to you.
What is not retried
If a backend rejects your request rather than failing to serve it — a malformed body, a parameter it will not accept — that answer comes straight back to you. Another backend would say the same thing, more slowly.
The short version: unavailability is retried, disagreement is not.
Streaming has a point of no return
A streamed request can still fail over while it is being set up. Once the backend has accepted it and the first bytes are on their way to you, the choice is committed — there is no way to silently switch backends mid-answer and no way to turn a partial answer into an HTTP error.
If a stream breaks after that, it ends carrying an error event rather than a status code. See Streaming.
Who pays for a failed attempt
Not you
Every attempt reserves an estimate against your balance and releases it the moment that attempt fails. A request that tried two backends before succeeding is charged once, for the one that worked.
The exception is a stream that broke partway: tokens were generated and delivered, so what was delivered is billed. See Billing.