Why your AI feature times out the request (and how queues fix it)

What actually happens on a slow synchronous call

A controller that calls an LLM provider directly is unremarkable code:

public function generate(Request $request)
{
    $response = Http::timeout(30)
        ->withToken(config('services.openai.key'))
        ->post('https://api.openai.com/v1/chat/completions', [
            'model' => 'gpt-4o',
            'messages' => [['role' => 'user', 'content' => $request->input('prompt')]],
        ]);

    return response()->json($response->json());
}

The problem isn't this code — it's what it costs while it's running. The PHP-FPM worker (or Octane worker) handling this request is occupied for the entire duration of the LLM call and cannot serve any other request in the meantime. A fast API call ties a worker up for milliseconds; this ties it up for however long the model takes to finish generating. Under real traffic, several users hitting this endpoint concurrently can exhaust your entire worker pool, and unrelated requests — ones that have nothing to do with AI — start queueing behind them or timing out.

Three separate timeouts that all need to agree

This is the part that produces genuinely confusing intermittent failures, because there isn't one timeout to check — there are several, configured in different places, and the request fails the moment any one of them is shorter than the actual response time:

• Http::timeout() — how long Laravel's HTTP client waits before giving up on the provider.
• PHP's max_execution_time — how long the PHP process itself is allowed to run before being killed, independent of what the code is waiting on.
• The web server or reverse proxy's timeout — Nginx's proxy_read_timeout defaults to 60 seconds; a load balancer in front of that may have its own, shorter default.

Set Http::timeout(30) correctly, and a request can still fail if proxy_read_timeout is set to 20 — the proxy gives up and returns a 504 before your own code's timeout would have. This tends to surface as "it works when I test it directly but fails through the actual site," because testing the endpoint directly (via artisan tinker, or hitting the app server without going through the proxy) skips the layer that's actually causing the failure.

The fix: don't make the user's request wait on the model

The real fix isn't tuning three timeout values to agree — it's not doing the slow work inside the request-response cycle at all. Dispatch a job, return immediately, and let the client find out when it's done:

class GenerateCompletion implements ShouldQueue
{
    public $tries = 3;
    public $timeout = 120; // this can be generous — it's not blocking a web worker

    public function __construct(
        public string $promptId
    ) {}

    public function handle(): void
    {
        $prompt = Prompt::findOrFail($this->promptId);

        $response = Http::timeout(90)
            ->withToken(config('services.openai.key'))
            ->post('https://api.openai.com/v1/chat/completions', [
                'model' => 'gpt-4o',
                'messages' => [['role' => 'user', 'content' => $prompt->text]],
            ]);

        $prompt->update([
            'result' => $response->json('choices.0.message.content'),
            'status' => 'completed',
        ]);
    }
}

public function generate(Request $request)
{
    $prompt = Prompt::create([
        'text' => $request->input('prompt'),
        'status' => 'processing',
    ]);

    GenerateCompletion::dispatch($prompt->id);

    return response()->json(['prompt_id' => $prompt->id], 202);
}

The web request now returns in milliseconds regardless of how long the model takes. The frontend polls a status endpoint, or — better for a snappier feel — you broadcast a completion event over a websocket connection (Laravel Reverb, Pusher) so the UI updates the moment the job finishes, rather than polling on an interval.

This is also just a more honest UI pattern: "generating..." with a spinner and no timeout risk beats a request that might 504 after 30 seconds of the user staring at a blank loading state.

If you genuinely need real-time streaming

Some UIs specifically want tokens appearing incrementally as the model generates them, ChatGPT-style — a queued job that returns everything at once doesn't fit that. Server-Sent Events (SSE) can stream a response as it arrives, but note this reintroduces the original problem in a different shape: an SSE connection held open for the duration of generation still occupies a worker connection for that whole time, and needs its own timeout configuration (both PHP and any reverse proxy in front of it) tuned for a long-lived connection rather than a normal quick request. Streaming solves the UX problem of "show the words as they arrive," not the underlying resource-holding problem — plan worker capacity and proxy timeouts for held-open connections deliberately if you go this route, rather than assuming it behaves like a normal fast request.

Retries need to expect more failures than a typical API

LLM provider APIs return 429 (rate limited) and transient 5xx errors more often than most internal or well-established third-party APIs — providers are handling enormous, spiky load, and this genuinely does happen in normal operation, not just as a rare edge case:

$response = Http::retry(3, 2000, throw: false)
    ->timeout(90)
    ->withToken(config('services.openai.key'))
    ->post('https://api.openai.com/v1/chat/completions', [...]);

Http::retry(times, sleepMilliseconds) retries automatically on failure with a delay in between — treat this as close to mandatory for any production LLM integration, not an optional hardening step.

The double-charge risk

If a job times out or the connection drops after the provider actually processed the request and returned a result — or after a job is retried following a genuine transient failure — you can end up calling the same expensive completion twice for what the user experiences as one request. Since LLM calls are billed per token, this is a real cost concern, not just a correctness one. Where the provider supports it, pass an idempotency key (OpenAI's API accepts one directly) so a retried request with the same key returns the original result instead of generating a new one and billing again.

Checklist before shipping an LLM-backed endpoint

• Does anything call the LLM provider synchronously inside a web request, rather than dispatching a job and returning immediately?
• Do Http::timeout(), PHP's execution time limit, and any reverse proxy timeout in front of the app all agree on a value long enough for the slowest realistic completion?
• Is Http::retry() configured, given how routinely providers return 429s and transient 5xx errors under normal load?
• If a request could plausibly be retried (client timeout, job retry), is an idempotency key in use where the provider supports one, to avoid a duplicate billed completion?