Background Jobs

Durable background jobs with the transactional outbox: a job is saved in the same Postgres transaction as the write that needs it, then runs from a queue, with retries, schedules and replay.

Work that must happen because a write happened (send a welcome email after sign-up, purge an organization's events after it's deleted, sync a record to another system) goes through the outbox. A job is a row saved in the same Postgres transaction as your write: if the write commits, the job exists and will run; if it rolls back, the job never existed. Nothing is lost between "saved" and "queued", and nothing runs for a write that didn't happen.

import { db } from 'void/db';
import { welcomeEmail } from '#/jobs/welcome-email.job.ts';
import { getOutbox } from '#/lib/outbox.server.ts';
await getOutbox().transaction(db, async (tx, enqueue) => {
const [row] = await tx.insert(user).values(input).returning();
await enqueue(welcomeEmail, { payload: { userId: row.id }, dedupeKey: `welcome:${row.id}` });
});

The kit uses @makerkit/workers-outbox, bound for you: you define jobs and enqueue them, nothing else.

How It Runs

  1. Enqueue: enqueue inserts a row in outbox.job inside your transaction.
  2. Nudge: once the transaction commits, the outbox sends the job's id to the outbox queue (queues/outbox.ts). The consumer claims and runs it within seconds.
  3. Sweep: every five minutes crons/outbox.ts recovers jobs whose worker crashed, runs any whose nudge was lost, and deletes finished jobs after their retention (7 days succeeded, 90 days dead or cancelled).
A transaction commits business data and an outbox job together. A queue nudge or sweep delivers the job; effects can repeat after a crash.Click to expand

The transaction covers the business write and job row only. It cannot roll back an email or an outside API call.

A job runs at least once: a crash between the work and its bookkeeping means it runs again. Write handlers so a second run does no harm (Jobs).

Job or Plain Queue?

UseWhen
An outbox jobThe work must follow a database write, and must not be lost or run without it: a purge after a delete, a notification for a committed change, a sync.
queues.<name>.send() (void/queues)Fire-and-forget work with no write to tie it to, where losing a message on a crash is acceptable. The kit's auth emails use the emails queue this way.
NeitherWork that must finish before the response: do it in the request.

Files

FileOwnerWhat
apps/web/config/outbox.config.server.tsyououtboxJobs: every job the outbox may run
apps/web/jobs/*.job.tsyouYour job definitions (any folder works)
apps/web/crons/<name>.tsyouA recurring job (Schedules)
apps/web/lib/outbox.server.tskitgetOutbox, defineOutboxSchedule, OutboxEnqueueError
apps/web/config/kit-outbox-jobs.server.tskitThe kit's own jobs (the daily cleanup's), spread into outboxJobs
apps/web/config/outbox.server.tskitThe binding: database, queue, deploy version, logger
apps/web/queues/outbox.ts, crons/outbox.tskitThe consumer and the sweep
packages/outbox (@kit/outbox)kitStrict enqueue, logging and redaction

Next: define and enqueue a job, run one on a schedule, handle failures, test it, deploy.