Rate limits, and the retries that make them worse

The provider rate-limited you, so you sent three more

A single line went into the integration, because a piece on this site about AI features timing out the request called it close to mandatory, and it is:

$response = Http::retry(3)
    ->withToken(config('services.openai.key'))
    ->post('https://api.openai.com/v1/chat/completions', [...]);

Fake a 429 behind that and count what leaves the machine. Three requests. Total elapsed: ten milliseconds. Then a RequestException.

That is not a retry. It is the original request, sent three times, inside the same window the provider just told you to back off from — and from the provider's side it is indistinguishable from you tripling your traffic the instant they asked you to stop. Rate limiters that use a sliding window count all three. You have made the thing you were recovering from worse, and you did it in ten milliseconds.

The second argument is the sleep, and it defaults to zero:

public function retry(array|int $times, Closure|int $sleepMilliseconds = 0, ?callable $when = null, bool $throw = true)

Http::retry(3) leaves it at zero. Http::retry(3, 100) is the shape almost everyone means, and almost nobody writes.

It also retries the failures that will never succeed

The third argument, $when, defaults to null, and null means retry everything. Everything includes the responses that are telling you the request itself is wrong.

Fake a 401 — an expired key, a project that lost access — and Http::retry(3, 10) sends it three times. Same for a 400 on a malformed payload. Three round trips, three identical rejections, and an exception at the end that says the same thing the first response said.

The distinction that matters is not "did it fail" but "would sending it again change anything":

Http::retry(3, 250, when: function (RequestException $e) {
    return in_array($e->response->status(), [429, 500, 502, 503, 504]);
})

Adding retry() changes the failure mode, even when it does not retry

This is the part that turns a hardening change into an incident.

A bare Http::post() that gets a 429 returns a Response object. $response->failed() is true, $response->status() is 429, and nothing is thrown. Code downstream that checks $response->successful() handles it.

Put retry() in front of the same call with more than one attempt, and a failed response throws RequestException instead, because the fourth argument, $throw, defaults to true. That is intended for the exhausted case — but it fires on the filtered case too. Give the when filter above a 400: it correctly declines to retry, sends exactly one request, and still throws. The filter controls retrying. It does not control throwing.

The boundary is tries > 1. Http::retry(1, 10) on a 429 returns the response quietly; Http::retry(2, 10) throws. So the upgrade from "no retries" to "a few retries" silently converts every handled failure in that call path into an uncaught exception.

If the calling code is built around inspecting responses, say so:

$response = Http::retry(3, 250, when: $retryable, throw: false)
    ->post('https://api.openai.com/v1/chat/completions', [...]);

if ($response->failed()) {
    // Same shape as before retry() was added.
}

Backoff, jitter, and the header nobody reads

Pass a closure as the delay and you get the attempt number and the exception, which is enough to do this properly. Exponential growth stops you hammering; jitter stops every worker in the fleet retrying on the same tick after a shared outage — without it, a hundred jobs that failed together retry together, and you have rebuilt the burst at a lower frequency.

The other half is the Retry-After header. Providers send it on a 429 to tell you exactly how long to wait. Laravel's HTTP client never reads it — there is no reference to that header anywhere in the client, and the delay it uses is only ever the one you supplied. It is reachable from the exception, so read it yourself and prefer it over your own guess:

Http::retry(4, function (int $attempt, RequestException $e) {
    $retryAfter = (int) ($e->response->header('Retry-After') ?: 0);

    if ($retryAfter > 0) {
        return $retryAfter * 1000;
    }

    return (2 ** $attempt) * 1000 + random_int(0, 1000);
}, when: $retryable)

Two details in that closure. The exception arrives with a full Response on it, so header(), status() and json() all work. And the closure runs once per gap, not once per attempt: four tries means it is called for attempts 1, 2 and 3.

The queue has its own retry, and its default is also "immediately"

Everything above is one HTTP call. The job wrapping it retries too, and the two multiply: three job attempts times four HTTP attempts is twelve calls to a metered endpoint for one dispatch.

ThrottlesExceptions is the right tool — it counts exceptions and opens a circuit once the count is reached, so the job stops calling the provider at all. Two things about it are easy to get wrong, and both show up the first time you run it.

backoff() takes minutes. Internally the value is multiplied by 60. ->backoff(5) releases the job 300 seconds later. And the default is 0, so a job with this middleware and no backoff() call is released with zero delay and picked straight back up — the same tight loop as Http::retry(3), one layer up.

Releases count against $tries. This is the one that quietly disables the middleware. Take a job with $tries = 3 and new ThrottlesExceptions(10, 600), dispatch it against a provider that is down, and watch the queue table: attempts 1, 2, 3, then failed_jobs. The throttle threshold is ten. The job never reaches it. The circuit breaker you installed never got a chance to open, and the job died on the ordinary retry limit instead.

Time-based attempts are the fix, because retryUntil() bypasses the $tries comparison entirely:

#[Tries(25)]
class GenerateCompletion implements ShouldQueue
{
    use Queueable;

    public function middleware(): array
    {
        return [(new ThrottlesExceptions(3, 600))
            ->backoff(5)
            ->failWhen(fn (Throwable $e) => $e instanceof RequestException
                && in_array($e->response->status(), [400, 401, 403, 422]))];
    }

    public function retryUntil(): DateTimeInterface
    {
        return now()->addMinutes(30);
    }
}

Run that against a failing provider and the shape is right: the first three failures are released 300 seconds apart, and from the fourth on the middleware short-circuits before handle() is called, releasing for roughly the full 600-second decay window without touching the API. failWhen() sends the terminal statuses straight to failed_jobs rather than grinding through thirty minutes of retries on a bad key — the same retryable-versus-terminal split as the HTTP layer, enforced at the layer that costs you wall-clock time.

One more thing to know about the throttle: the cache key is the job's class name by default, so every instance shares one bucket. Two jobs hitting the same provider need ->by('openai') on both to share a limit, and a job that must be throttled per tenant needs the tenant in that key.

Make the spend visible, or you will find out on the invoice

Every layer above can triple your call volume, and none of them log a number you can add up. The ResponseReceived event fires once per HTTP attempt — including retried ones — which makes it the only place that sees the true count:

Event::listen(function (ResponseReceived $event) {
    if (! str_contains($event->request->url(), 'api.openai.com')) {
        return;
    }

    Log::channel('llm')->info('provider call', [
        'status' => $event->response->status(),
        'tokens' => $event->response->json('usage.total_tokens'),
    ]);
});

Fake a sequence of 429, 429, 200 behind Http::retry(3, 5) and that listener fires three times, with the token usage on the third. Three log lines for one dispatch. That ratio — attempts per successful completion — is the number to alert on, because it goes bad long before anything reaches failed_jobs, and it is what the bill is actually counting.

Worth pairing with an idempotency key where the provider supports one: a retry that fires after the provider already generated a completion is a second billed generation for one logical request.

What to change

Open the file with your provider call in it and check four things. Does every retry() have a sleep argument and a when filter, or is it still retry(3) on its own? Did adding retry() turn a handled failure response into an unhandled exception somewhere downstream, and does that path need throw: false? Does the delay closure read Retry-After before falling back to your own backoff? And on the job side, is $tries high enough — or replaced by retryUntil() — that ThrottlesExceptions can actually reach its threshold before the job fails?

Then add the ResponseReceived listener and look at attempts per completion for a day. If it is not close to one, something above is retrying things that were never going to succeed, and you are paying for it.