Loading...

This is taking longer than expected.

Coding standards

Queues and Time Budget

These are the standards the analyzer applies. Every rule states what it checks and why it exists.

All standards

Nothing Runs Longer Than Five Seconds

No unit of work runs longer than five seconds: a web request, a Livewire action, a job, an Artisan command or a migration. Work that can run past five seconds is not done inline — it goes to the queue, as the Queues and Time Budget standard describes.
The budget is set before the code is written, not discovered after it fails. A request that runs for minutes ends in a 502 or a gateway timeout, a migration killed mid-run is retried from the start, and a run cut off halfway is left "in progress" forever. Every one of those costs the person who asked for the work a second attempt.
Five seconds is the ceiling, not the target. Measure the work over the largest data the project will hold, not over today's.
The analyzer does not check this standard yet: no rule here has a processor, so a clean analyzer run proves nothing about it. It is followed by whoever plans and builds the work.

What Can Outrun the Budget

Work can outrun the budget when its cost grows with the data, the files or the external services it reaches:

  • a loop over rows that does the work itself, one row at a time;
  • copying, moving, hashing or deleting files on a disk or S3;
  • calculating over many rows, or building a report or an export;
  • calling an external service — an API, a mail provider, a model — once per row, or a single call whose timeout is not set inside the budget. A single call that sets one (Http::timeout()) and whose answer the person needs to go on stays inline.

Bounded work stays inline: a fixed list of ids, creating one record or correcting one text, and a single set-based statement (one update(), delete() or insert … select), which is one round trip however many rows it reaches, unless the table is large enough for the statement itself to hold locks for a long time.
When it is unclear whether something is bounded, it is not. A cost that grows with the data grows past any limit given enough data.

Heavy Work Goes to the Queue and the Request Answers at Once

Work that can outrun the budget is dispatched to the queue, and the request that started it answers immediately, without waiting for the result. The answer says the work has started; it never blocks until it ends.
The entry point — a controller, a Livewire action, a command — calls a model method, and the model dispatches the work, as every other standard expects of an entry point.

// Correct — the action starts the work and returns at once
public function export(): void
{
    $this->authorize('export', $this->project);
    $this->export = $this->project->startOrderExport();
}

// Incorrect — the action waits for every order
public function export(): void
{
    $this->authorize('export', $this->project);
    foreach ($this->project->orders as $order) {
        $this->project->exportOrder($order);
    }
}

A Large Job Is a Batch of Small Jobs

A long job times out too. A worker kills a job that runs past its timeout and retries it from the start, so work whose cost grows with the data is never one job.
The work is split into many small jobs dispatched as one batch (Bus::batch()):

  • A coordinator splits the work, inside the budget too. The request dispatches the batch with one coordinator job, so splitting the work happens on the queue and never in the request. The coordinator never walks the rows: it reads the lowest and the highest id once and adds one small job per id range to its own batch with $this->batch()->add(). When the ranges are too many to add inside the budget, it adds a slice of them and then the coordinator of the next slice, so the split is divided the same way the work is.
  • Each small job stays inside the five-second budget, which also keeps it far under the worker timeout of the target environment. Use ranges of 20 to 50 ids for file, S3 or external-service work and a few hundred for plain SQL; the number is whatever keeps one job inside the budget in the slowest case. Declare $timeout on the job, below the worker's, and keep the connection's retry_after above it.
  • The jobs are spread so they do not starve the queue. Put them on a queue of their own, or spread them with delay(), instead of releasing thousands at once ahead of every other job.
  • Only the coordinator dispatches. A small job never dispatches new jobs on its own and never dispatches itself again; the coordinator adds every range.
  • A closing step collects the result. The batch's then() builds the final result — the file, the summary, the notification — and catch() records the failure. Both callbacks are serialized, so they are static closures that capture only plain values, never $this. When collecting the result can itself outrun the budget, the closing step dispatches one more job rather than doing it inline.
// Correct — the model starts a batch that a coordinator fills
public function startOrderExport(): OrderExport
{
    $export = $this->orderExports()->create();
    $exportId = $export->id;

    $batch = Bus::batch([new PlanOrderExport($this->id, $exportId, null)])
        ->then(static function (Batch $batch) use ($exportId) {
            OrderExport::find($exportId)?->finish();
        })
        ->catch(static function (Batch $batch, Throwable $error) use ($exportId) {
            OrderExport::find($exportId)?->markFailed($error->getMessage());
        })
        ->onQueue('exports')
        ->dispatch();

    $export->update(['batch_id' => $batch->id]);

    return $export;
}

// The coordinator, inside PlanOrderExport::handle() — a slice of ranges, then the next coordinator
$orders = Order::query()->where('project_id', $this->projectId);
$last = $orders->clone()->max('id');
if ($last === null) {
    return;
}

$from = $this->from ?? $orders->clone()->min('id');
$until = min($from + 50 * 200 - 1, $last);
$jobs = collect(range($from, $until, 50))->map(function (int $start) use ($until) {
    return new ExportOrders($this->projectId, $this->exportId, $start, min($start + 49, $until));
});
$this->batch()->add($jobs->all());

if ($until < $last) {
    $this->batch()->add([new PlanOrderExport($this->projectId, $this->exportId, $until + 1)]);
}

Every Job Is Idempotent

A job can be retried, cut off or run twice, and the result is the same every time. Re-check what is already done before acting, never duplicate a row or a file, and delete an original only after its copy is verified.
Each job records its progress — the batch does it for you when the work runs as one — so a stuck run can be told apart from a slow one. Do not resume from a stored last-id cursor: each job is handed its range when it is added to the batch, and a cursor turns a retry into a skipped range.

The Person Sees the Work Advance

While the batch runs, the screen that started it shows a progress bar and keeps it current by polling the batch's state with Livewire (wire:poll), through a #[Computed] property that reads the batch the record points at. The record reads as running from the moment it is created until the batch's then() or catch() marks it, and the poll stops then. When a coordinator adds the work in slices, the batch's own progress counts only the jobs added so far and falls back each time a slice is added: record the total once, from the first coordinator's lowest and highest id, on the record the batch belongs to, and measure progress against it.

  • When the batch ends, the screen says the work is done and where its result is.
  • When it fails, the screen says so, in words the person understands, and what they can do next.
  • Closing the screen does not stop the work. The batch runs on the queue whether anybody is watching, and the record it belongs to keeps its state, so returning to the screen shows where it got to.
#[Computed]
public function exportState(): ?string
{
    return $this->export?->state();
}

#[Computed]
public function exportProgress(): ?int
{
    return $this->export?->progress();
}
<div @if ($this->exportState === 'running') wire:poll.2s @endif>
    @if ($this->exportState === 'running')
        <progress class="w-full" max="100" value="{{ $this->exportProgress }}"></progress>
    @elseif ($this->exportState === 'finished')
        <p class="text-positive-600">{{ __('The export is ready.') }}</p>
    @elseif ($this->exportState === 'failed')
        <p class="text-negative-600">{{ __('The export failed. Start it again.') }}</p>
    @endif
</div>

A Worker Must Be Running

The work is not done when the request ends. It is done when a queue worker has run every job of the batch.

  • A queue worker must be running in every environment the work reaches. php artisan queue:restart only tells running workers to reload; it does not start one.
  • With QUEUE_CONNECTION=sync the jobs run inline inside the request, which brings back the timeout this standard exists to prevent. Use a real queue connection.
  • Batches need the job_batches table. Create it with the project's migration before the first batch is dispatched.

Migrations Follow Their Own Rule

A migration's deploy waits for it, so its own work stays inside the budget, and Heavy Data Work Goes Through the Queue in the Laravel Migrations standard is the form its heavy work takes. That rule is the particular case of this standard and it stays as it is: a migration selects only the ids itself and dispatches static closures, one per chunk, and never a Job class or a coordinator. Walking those ids is the one split this standard lets run outside a coordinator, and how far it may go is that rule's to bound. Each closure is a job, so it stays inside the five seconds as well as under the worker timeout. That rule asks the rest of what applies to a migration in its own words — small chunks, idempotent work, a spread dispatch and a running worker. A migration has no screen and no closing step.

Weblabor Logo Developed by Weblabor