Why your export still runs out of memory after you add chunk()

The export that finished and was wrong

The nightly CSV export died with Allowed memory size of 536870912 bytes exhausted. You found the cause in about a minute — Order::all() over 400,000 rows — and you fixed it the way everyone fixes it:

$rows = [];

Order::where('exported', false)->chunk(500, function ($orders) use (&$rows) {
    foreach ($orders as $order) {
        $rows[] = $this->toCsvRow($order);
        $order->update(['exported' => true]);
    }
});

The job stopped crashing. Everyone moved on. Three weeks later finance asked why the export had 380,000 rows instead of 400,000.

Both bugs are still in that code. It still accumulates every row in $rows, so you raised the ceiling rather than removed it. And it skips records — roughly one in every two — because of how chunk() walks the table.

chunk() is paginating a table you are editing underneath it

chunk() is offset pagination. Each iteration runs a fresh query with a new OFFSET, and the framework enforces an ORDER BY so the pages are at least stable relative to each other:

$results = $this->offset($offset)->limit($limit)->get();

Page 1 is OFFSET 0 LIMIT 500. Page 2 is OFFSET 500 LIMIT 500. That is correct only if the result set does not move between queries.

Yours moves. The query filters on exported = false, and the loop body sets exported = true. After the first chunk, 500 rows no longer match the WHERE clause, so they drop out of the result set entirely. Every remaining row shifts 500 positions to the left. Then the framework asks for OFFSET 500 — and skips the 500 rows that just slid into positions 0 through 499.

You process 500, skip 500, process 500, skip 500. Half the table, and not a single error to show for it.

The docs are explicit about this: if you filter on a column you will also update while iterating, chunk() "could lead to unexpected and inconsistent results". The word doing the work there is inconsistent. Nothing throws. The job exits zero. The only symptom is a number that is wrong in a report nobody reconciles for three weeks.

This is not limited to updates. Deleting rows inside the callback does the same thing, and so does any change to the primary key or the foreign keys the query depends on.

chunkById() walks by key, not by position

chunkById() drops offsets entirely. It remembers the last primary key it saw and asks for rows greater than it:

Order::where('exported', false)
    ->chunkById(500, function ($orders) {
        foreach ($orders as $order) {
            $order->update(['exported' => true]);
        }
    }, column: 'id');

Now "where I got to" is a value in the data, not a position in a result set. Rows leaving the result set behind the cursor cannot shift anything in front of it. The scan is also faster on a large table, because the database seeks an indexed key instead of counting and discarding 300,000 rows to satisfy OFFSET 300000.

There is a trap that comes with it, and it bites the first time you write a query with an OR. chunkById() appends its own where condition to your query. If your conditions are not grouped, you get WHERE a = 1 OR b = 2 AND id > 500, and SQL operator precedence binds that AND to the b = 2 branch only. The keyset condition applies to half your query, and the chunker loops over the same rows forever.

Group your own conditions in a closure and the appended condition lands where you meant it to:

Order::where(function ($query) {
    $query->where('status', 'pending')->orWhere('status', 'retrying');
})->chunkById(500, function ($orders) {
    $orders->each->update(['exported' => true]);
}, column: 'id');

The same applies to lazyById(), for the same reason.

cursor() runs one query, and PDO is still holding all of it

The instinct after learning all this is to reach for cursor(), because the description sounds like exactly what an export needs: one query, a generator, one model in memory at a time.

All of that is true, and it will still exhaust your memory limit.

cursor() controls hydration, not transfer. Only one Eloquent model exists in PHP at a time, but the underlying PDO driver buffers the complete raw result set in memory before your first foreach iteration runs. You removed the cost of 400,000 hydrated models and kept the cost of 400,000 raw rows. On a wide table, the raw rows were the larger number.

cursor() carries a second restriction that matters more for exports than people expect: it cannot eager load relationships. It holds one model, and eager loading needs the whole set to batch the relationship query. So the moment your CSV includes $order->customer->name, a cursor() export becomes one query per row — a textbook N+1, now wearing a performance fix as a disguise.

cursor() earns its place on a narrow query over a bounded set where you need a single round trip. It is the wrong default for an export.

lazy() is the one you wanted, with the same caveat as chunk()

lazy() runs chunked queries like chunk(), but flattens them into a LazyCollection you iterate as one stream, so you write a flat loop instead of nesting your logic inside a callback. The default chunk size is 1000. Because it executes a new query per chunk, PDO only ever buffers 1000 rows, and because it yields models one at a time, only one is hydrated. It also eager loads normally.

Read the implementation and you will see it calls forPage(). That is offset pagination — which means lazy() has exactly the same skipping bug as chunk(). If you mutate the column you filter on, use lazyById():

Order::where('exported', false)
    ->lazyById(1000, column: 'id')
    ->each(function (Order $order) {
        $order->update(['exported' => true]);
    });

That fixes the reading half. The accumulating half is still there as long as you build $rows in memory. Hand each row to the client as you produce it instead:

return response()->streamDownload(function () {
    $handle = fopen('php://output', 'w');

    Order::with('customer')->lazyById(1000, column: 'id')->each(
        fn (Order $order) => fputcsv($handle, $this->toCsvRow($order))
    );

    fclose($handle);
}, 'orders.csv');

Peak memory is now one chunk plus one row, whether the table holds 400,000 rows or four million.

What to change

Open the exports and backfills you have already "fixed" and check two things in each.

First, does the loop body write to a column the query filters on? If yes, you have been silently skipping records since the day it shipped — switch to chunkById() or lazyById() and reconcile the counts before you assume the old runs were fine. Second, does anything accumulate across iterations? An array, a string being concatenated, a collection you push() to. Chunked reads do nothing for a buffer that only grows; stream the output instead.

Then check for orWhere in any keyset-chunked query and wrap those conditions in a closure.

If you need a rule for the next one: reach for lazyById() by default, chunkById() when you genuinely want to act on batches, and cursor() only when you have measured that a single round trip matters and you are loading no relationships.