# Rate Limiting and Load Shedding

> Protect your API with Redis-backed rate limits, honest 429 responses, load shedding, circuit breakers and timeouts before traffic takes it down.

- Author: Syed Wajeeh Haider
- Published: Aug 25, 2026
- Series: Scaling to 1M Requests, part 7 of 10
- Category: Reliability
- Tags: Rate Limiting, Redis, Reliability, API Design
- Reading time: 3 min
- URL: https://syedwajeehhaider.com/rate-limiting-and-load-shedding

Capacity is finite, and at a million requests some of that traffic will be retries, scrapers, buggy clients and bursts you didn't plan for. A system that tries to serve everything under overload ends up serving nothing. Rate limiting decides who gets capacity; load shedding decides what to drop when there isn't enough to go around.

- **429**: Too Many Requests: the honest answer
- **503**: Service Unavailable, for shed load
- **1**: Round trip per check with a Lua script

## Choosing an algorithm

- **Fixed window**: count requests per minute. Simple, but it allows double bursts at window edges.
- **Sliding window**: smooths the edges by weighting the previous window. A good default for per-user API quotas.
- **Token bucket**: tokens refill at a steady rate up to a capacity, and each request spends one. It allows short bursts while enforcing a long-term average, which matches how real clients behave.

## A Redis token bucket

With many API instances, the limit has to live somewhere shared. Redis is the usual choice, and a Lua script keeps the read-refill-spend sequence **atomic**, so two concurrent requests can never spend the same token.

*token-bucket.lua*

```lua
-- KEYS[1] = bucket key
-- ARGV: capacity, refill rate (tokens/sec), now (ms), cost
local capacity = tonumber(ARGV[1])
local rate = tonumber(ARGV[2])
local now = tonumber(ARGV[3])
local cost = tonumber(ARGV[4])

local bucket = redis.call('HMGET', KEYS[1], 'tokens', 'ts')
local tokens = tonumber(bucket[1]) or capacity
local ts = tonumber(bucket[2]) or now

tokens = math.min(capacity, tokens + ((now - ts) / 1000) * rate)
local allowed = 0
if tokens >= cost then
  tokens = tokens - cost
  allowed = 1
end

redis.call('HSET', KEYS[1], 'tokens', tokens, 'ts', now)
redis.call('PEXPIRE', KEYS[1], math.ceil((capacity / rate) * 1000))
return { allowed, math.floor(tokens) }
```

*Express middleware using ioredis*

```ts
import Redis from 'ioredis';
import { readFileSync } from 'node:fs';

const redis = new Redis(process.env.REDIS_URL!);
redis.defineCommand('takeToken', {
  numberOfKeys: 1,
  lua: readFileSync('./token-bucket.lua', 'utf8'),
});

const CAPACITY = 100; // burst size
const RATE = 10; // tokens per second

export async function rateLimit(req, res, next) {
  const key = 'rl:' + (req.user?.id ?? req.ip);
  const [allowed, remaining] = await (redis as any).takeToken(
    key, CAPACITY, RATE, Date.now(), 1,
  );

  res.set('RateLimit-Limit', String(CAPACITY));
  res.set('RateLimit-Remaining', String(remaining));

  if (!allowed) {
    res.set('Retry-After', String(Math.ceil(1 / RATE)));
    return res.status(429).json({ error: 'rate_limited' });
  }
  next();
}
```

> **Pick the right key:** Limit authenticated traffic **per user or API key**, and anonymous traffic **per IP**. Keep a separate, stricter bucket for expensive endpoints like search, login and exports. A single global limit punishes good users for the behaviour of bad ones.

## Honest responses

Return `429 Too Many Requests` with a `Retry-After` header, and make your own clients (web, React Native, Flutter) honour it with backoff. Silent drops and generic 500s cause clients to retry immediately, which multiplies the load you were trying to avoid.

## Load shedding

Rate limits protect you from individual clients. Load shedding protects you from everyone at once. When the service is already saturated, rejecting some requests quickly beats serving all of them slowly. In Node.js, event loop delay is an excellent saturation signal:

```ts
import { monitorEventLoopDelay } from 'node:perf_hooks';

const loop = monitorEventLoopDelay({ resolution: 20 });
loop.enable();
setInterval(() => loop.reset(), 10_000); // look at recent behaviour only

app.use((req, res, next) => {
  const p99ms = loop.percentile(99) / 1e6; // nanoseconds to ms
  if (p99ms > 200 && req.path !== '/health') {
    res.set('Retry-After', '5');
    return res.status(503).json({ error: 'overloaded' });
  }
  next();
});
```

Shed the cheap-to-reject, low-priority traffic first: background syncs, prefetches and analytics beacons. Protect checkout and login until the very end.

## Circuit breakers, timeouts and bulkheads

- **Timeouts** on every outbound call. `fetch(url, { signal: AbortSignal.timeout(2000) })` means a slow dependency costs you two seconds, not a stuck connection.
- **Circuit breakers** stop calling a dependency after repeated failures, fail fast for a cool-down period, then let a trial request through. Libraries like `opossum` implement this for Node.js.
- **Bulkheads** give each dependency its own connection pool or concurrency limit, so a slow payments API cannot exhaust the pool your product pages need.

> A fast failure is a feature. A slow failure is an outage that spreads.

## Key takeaways

- Use a token bucket in Redis, updated atomically with a Lua script.
- Key limits by user, API key or IP, with stricter buckets for expensive endpoints.
- Answer with `429` plus `Retry-After`, and make clients back off.
- Shed low-priority load early, and put timeouts and circuit breakers on every dependency.

Next we look at a platform that scales for you, up to a point: **Serverless at Scale: When AWS Lambda Fits**.