Node.js Under Load: Event Loop, Clustering, Backpressure

Node.js handles huge numbers of concurrent connections because it never waits: while one request is waiting on the database, the same thread serves others. That model has one rule you cannot break. Never block the event loop. Everything in this part is about keeping that single thread free, and then running enough of them.

Measure event-loop delay first

Event-loop delay is the time a callback waits before it can run. When it climbs, every request slows down together, regardless of which endpoint caused it. Node ships a histogram for exactly this in perf_hooks, and it is cheap enough to run in production and export as a metric.

import { monitorEventLoopDelay } from 'node:perf_hooks';

const histogram = monitorEventLoopDelay({ resolution: 20 });
histogram.enable();

setInterval(() => {
  // Values are in nanoseconds
  const p99 = histogram.percentile(99) / 1e6;
  const max = histogram.max / 1e6;
  console.log(JSON.stringify({ metric: 'event_loop_delay_ms', p99, max }));
  histogram.reset();
}, 10_000).unref();

A healthy service shows single-digit milliseconds. Sustained delays in the tens or hundreds of milliseconds mean something synchronous is hogging the thread.

Move CPU work off the main thread

The usual suspects are large JSON.parse or JSON.stringify calls, image processing, PDF generation, crypto with synchronous APIs, heavy regular expressions and big in-memory sorts. The options, in order of preference:

• —Do less: paginate, stream, or return smaller payloads.
• —Use async APIs: for example crypto.scrypt instead of crypto.scryptSync.
• —Offload to a worker thread for CPU-bound work that must happen in-process.
• —Offload to a queue for anything that does not need to finish before the response (covered in part 6).

// resize.worker.ts
import { parentPort, workerData } from 'node:worker_threads';
import { heavyTransform } from './transform';

parentPort!.postMessage(heavyTransform(workerData));

// route handler
import { Worker } from 'node:worker_threads';

function runInWorker<T>(file: string, data: unknown): Promise<T> {
  return new Promise((resolve, reject) => {
    const worker = new Worker(file, { workerData: data });
    worker.once('message', resolve);
    worker.once('error', reject);
  });
}

Creating a worker per request is fine for occasional jobs; for sustained load, keep a small pool of workers and reuse them.

Use every core

One Node process uses one core for JavaScript. On a four-core instance, a single process leaves most of the machine idle. Either run one process per core with node:cluster or a process manager such as PM2 in cluster mode, or, in containers, run one process per container and let the orchestrator schedule as many containers as there are cores. Pick one approach; running cluster mode inside many small containers just adds overhead.

# PM2: one process per CPU core, zero-downtime reloads
pm2 start dist/server.js -i max --name api
pm2 reload api

Reuse outbound connections

Calling other services is often where latency hides. Opening a new TCP and TLS connection for every outbound request adds round trips and can exhaust ephemeral ports under load. Reuse connections with keep-alive. The built-in fetch in modern Node is backed by undici, which pools connections by default; if you use the http module or older clients, pass an agent with keepAlive: true and a sensible maxSockets.

Respect backpressure with streams

When you move large data, such as exports or proxied files, read and write it as a stream rather than loading it into memory. Streams have backpressure built in: if the client reads slowly, the source pauses. Use pipeline so errors and cleanup are handled correctly on both ends.

import { pipeline } from 'node:stream/promises';
import { createGzip } from 'node:zlib';

app.get('/export.csv.gz', async (req, res) => {
  res.setHeader('Content-Type', 'application/gzip');
  // exportRowsAsCsvStream returns a Readable that queries in batches
  await pipeline(exportRowsAsCsvStream(req.query), createGzip(), res);
});

Key takeaways

• —Track event-loop delay as a first-class metric; it predicts slowdowns across every endpoint.
• —Keep CPU-heavy work off the main thread with async APIs, worker threads or queues.
• —Run one process per core, via cluster or PM2, or one process per container.
• —Reuse outbound connections with keep-alive, and put a timeout on every call.
• —Stream large payloads with pipeline so backpressure protects your memory.

Next up, part 6: Queues and Async Work, where slow tasks leave the request path entirely.