Queues and Async Work

At a million requests, the slowest thing in your request handler sets the pace for everything else. Sending an email, resizing an image, calling a partner's webhook: none of these need to finish before the user gets a response. The moment you stop doing them inline, your p99 drops, your servers hold fewer open connections, and a slow third party stops being your outage.

What belongs off the request path

A simple rule: if the user doesn't need the result to see the next screen, it goes on a queue. The request handler validates input, writes the minimum to the database, enqueues a job and returns. A pool of workers picks jobs up at whatever rate your infrastructure can sustain.

• —Notifications: transactional email, push notifications, SMS.
• —Media: image resizing, video transcoding, PDF generation.
• —Integrations: outbound webhooks, CRM syncs, payment reconciliation.
• —Fan-out: updating feeds, search indexes and analytics after a write.

The 202 Accepted pattern

Instead of 200 OK with the finished result, the API answers 202 Accepted with an ID the client can poll, or it pushes the result later over a WebSocket or push notification. Here is the pattern with BullMQ on Redis, a solid choice for Node.js services:

import { Queue } from 'bullmq';

export const connection = { host: process.env.REDIS_HOST, port: 6379 };
export const mediaQueue = new Queue('media', { connection });

app.post('/uploads', async (req, res) => {
  const upload = await db.uploads.create({
    userId: req.user.id,
    key: req.body.key,
    status: 'pending',
  });

  await mediaQueue.add(
    'resize',
    { uploadId: upload.id },
    {
      jobId: `resize-${upload.id}`, // same upload, same job: no duplicates
      attempts: 5,
      backoff: { type: 'exponential', delay: 1000 },
      removeOnComplete: 1000,
    },
  );

  res.status(202).json({ id: upload.id, status: 'pending' });
});

import { Worker } from 'bullmq';

const worker = new Worker(
  'media',
  async (job) => {
    const upload = await db.uploads.findById(job.data.uploadId);
    if (upload.status === 'done') return; // already processed: safe to retry

    await resizeImage(upload.key);
    await db.uploads.update(upload.id, { status: 'done' });
  },
  { connection, concurrency: 10 },
);

worker.on('failed', (job, err) => {
  logger.error({ jobId: job?.id, err }, 'media job failed');
});

SQS or BullMQ?

BullMQ gives you delayed jobs, priorities, rate-limited workers and a friendly API, but you run and scale Redis yourself. Amazon SQS is fully managed and scales without capacity planning, which makes it the default on AWS when throughput is high and job semantics are simple. Many teams use both: SQS between services, BullMQ inside a single Node.js service.

Idempotency is not optional

Every queue you will use at scale delivers at least once. Workers crash after doing the work but before acknowledging it, networks drop acknowledgements, and timeouts expire. The job will run twice eventually, so design for it:

• —Derive a stable idempotency key from the business event, such as the upload ID or order ID, not from a random value.
• —Check state before acting, and make the final write conditional (for example UPDATE ... WHERE status = 'pending').
• —For external calls, pass the idempotency key to providers that support it, as most payment APIs do.

Retries, backoff and jitter

Retrying immediately turns a struggling dependency into a dead one. Use exponential backoff so each retry waits longer, and add jitter so thousands of failed jobs don't retry in lockstep:

// "Full jitter": random delay between 0 and the exponential cap
const retryDelay = (attempt: number, base = 500, cap = 30_000) =>
  Math.random() * Math.min(cap, base * 2 ** attempt);

Dead-letter queues

Some jobs will never succeed: malformed payloads, deleted records, bugs. After a fixed number of attempts, move them to a dead-letter queue instead of retrying forever. In SQS that's a redrive policy on the source queue:

{
  "deadLetterTargetArn": "arn:aws:sqs:us-east-1:123456789012:media-dlq",
  "maxReceiveCount": 5
}

Alert on DLQ depth, not on individual failures. A DLQ that grows steadily is a bug report; a DLQ that stays empty means the pipeline is healthy.

Key takeaways

• —If the user does not need the result right now, enqueue it and return 202 Accepted.
• —Assume every job runs more than once, and make handlers idempotent.
• —Use exponential backoff with jitter, a retry limit and a dead-letter queue.
• —Scale workers on queue depth and job age, independently of your API servers.

Queues absorb bursts, but they don't stop an abusive client or a traffic spike from overwhelming the front door. Next up: Rate Limiting and Load Shedding.