Skip to main content
Stackbastion

Draft, not yet published

Background jobs and queues: what AI app builders don't set up for you

7 July 2026· 6 min read · by Stackbastion

A user signs up and your app sends a welcome email, resizes their avatar, and calls a third-party API, all inside the same web request. Most of the time it works. Then one day the email provider is slow, the request hangs for 30 seconds, and the user gets an error even though their account was created fine. That’s what happens when slow work runs in the request instead of behind it.

Why this happens

When an AI tool builds a feature, it puts everything in one place: the route handler. Receive the request, do all the work, send the response. It reads clean and it demos fine, because on your laptop the email sends in 200 milliseconds.

Production is different. That email call now crosses the internet to a provider that occasionally takes 10 seconds. Your web server holds the connection open the whole time. Three things go wrong:

  • Timeouts. Most hosts cut off a request after 30 or 60 seconds. Slow work gets killed halfway, and the user sees a 500.
  • Lost work. If the email fails, the whole request fails, even though the signup itself succeeded. Or the account is made but the welcome email silently never sends. There’s no retry.
  • A blocked server. Each hanging request ties up a worker. Enough of them and your whole app stops responding, because every worker is stuck waiting on a slow API.

The fix is old and boring: do the slow part after you’ve answered the user. Save “send welcome email to user 812” as a job, return the response immediately, and let a separate worker pick the job up and run it. If it fails, the worker retries. AI builders almost never set this up, because the naive version works well enough to ship.

How to fix it

You don’t need Redis or a hosted queue service to start. If you already run Postgres, it can be your queue. A single table plus FOR UPDATE SKIP LOCKED gives you a real job queue that survives restarts and lets multiple workers run side by side without stepping on each other.

Step 1: the jobs table

CREATE TABLE jobs (
  id          bigserial PRIMARY KEY,
  type        text        NOT NULL,
  payload     jsonb       NOT NULL,
  run_at      timestamptz NOT NULL DEFAULT now(),
  attempts    int         NOT NULL DEFAULT 0,
  max_attempts int        NOT NULL DEFAULT 5,
  locked_at   timestamptz,
  created_at  timestamptz NOT NULL DEFAULT now()
);

-- Fast lookup for the next job that's due and not locked
CREATE INDEX idx_jobs_ready ON jobs (run_at) WHERE locked_at IS NULL;

Step 2: enqueue instead of doing the work inline

In your signup handler, don’t send the email. Just record that it needs sending, then respond:

// Inside the request: fast, no external calls
await db.query(
  `INSERT INTO jobs (type, payload) VALUES ($1, $2)`,
  ["send_welcome_email", { userId: user.id, email: user.email }]
);
return res.status(201).json({ ok: true }); // user gets an instant response

Step 3: a worker that claims and runs jobs

Run this as a separate process. The magic is FOR UPDATE SKIP LOCKED: it grabs one due job and locks its row, and any other worker running at the same time simply skips that row and takes the next one. No two workers ever run the same job.

async function claimAndRun(db, handlers) {
  const { rows } = await db.query(`
    UPDATE jobs
    SET locked_at = now(), attempts = attempts + 1
    WHERE id = (
      SELECT id FROM jobs
      WHERE locked_at IS NULL AND run_at <= now()
      ORDER BY run_at
      FOR UPDATE SKIP LOCKED
      LIMIT 1
    )
    RETURNING *;
  `);

  const job = rows[0];
  if (!job) return; // nothing to do right now

  try {
    await handlers[job.type](job.payload);
    await db.query(`DELETE FROM jobs WHERE id = $1`, [job.id]); // done
  } catch (err) {
    console.error(`Job ${job.id} (${job.type}) failed:`, err.message);
    if (job.attempts >= job.max_attempts) {
      // Give up, but keep the row for inspection instead of losing it
      await db.query(
        `UPDATE jobs SET locked_at = NULL, run_at = 'infinity' WHERE id = $1`,
        [job.id]
      );
    } else {
      // Back off: retry later, with the delay growing each attempt
      const backoffSeconds = Math.min(3600, 2 ** job.attempts * 10);
      await db.query(
        `UPDATE jobs SET locked_at = NULL, run_at = now() + ($1 || ' seconds')::interval WHERE id = $2`,
        [backoffSeconds, job.id]
      );
    }
  }
}

// Poll a few times a second
const handlers = {
  send_welcome_email: async ({ email }) => { /* call your email API */ },
};
setInterval(() => claimAndRun(db, handlers).catch(console.error), 500);

That’s a working queue with retries and exponential backoff, in one table and one loop. Failed jobs back off and try again, and after max_attempts they get parked with run_at = 'infinity' so you can look at them instead of losing them.

When to reach for a dedicated queue

If you outgrow this, BullMQ (backed by Redis) gives you scheduling, priorities, and a dashboard for a few lines of setup:

import { Queue } from "bullmq";
const emailQueue = new Queue("email", { connection: { host: "127.0.0.1", port: 6379 } });
await emailQueue.add("welcome", { userId: 812 }, { attempts: 5, backoff: { type: "exponential", delay: 10000 } });

But don’t add Redis just to send a welcome email. Start with the Postgres table. Move up only when volume or scheduling needs force it. That’s the KISS answer.

Or, we do it for you

Moving slow work off the request path, with retries and a worker that actually stays up, is standard in every production setup we run. If your app does emails, image processing, or third-party calls inside the request, get a free audit and we’ll show you what’s at risk.

FAQ

Is a Postgres queue really good enough for production?

For most small apps, yes. SKIP LOCKED is a proven pattern that handles thousands of jobs an hour without breaking a sweat. You get retries, durability, and no extra infrastructure. Reach for Redis-based tools when you need very high throughput or complex scheduling.

What happens to jobs if the worker crashes mid-run?

The row stays locked with a locked_at timestamp. Add a small sweeper that clears locks older than a few minutes so those jobs become claimable again. That way a crashed worker doesn’t strand its job forever.

Should the worker run in the same process as my web server?

Keep it separate if you can. A separate process means a stuck job can’t tie up the web server, and you can scale workers independently. On a small setup, a second process on the same machine is fine.

How do I avoid running the same job twice?

Make your handlers idempotent, meaning running one twice does no harm. Check “did I already send this email?” before sending, or use a unique key. Any queue can deliver a job twice after a crash, so idempotency is your real safety net.

Related posts