Caching Layers: CDN, Redis, and Memory

At a million requests, most of your traffic is asking the same questions over and over: the same product page, the same feed, the same config. Caching turns repeated work into a lookup. Done well, it is the single biggest multiplier on capacity you have. Done carelessly, it serves the wrong data to the wrong user. The trick is to cache in layers and be deliberate about each one.

Layer one: the CDN

A CDN such as CloudFront serves responses from edge locations close to users, so cached requests never reach your origin at all. Static assets are the obvious win, but public API responses and server-rendered pages can be cached too, as long as they are the same for everyone.

You control CDN behaviour with the Cache-Control header. max-age applies to browsers, s-maxage applies to shared caches like the CDN, and stale-while-revalidate lets the edge serve a slightly old copy instantly while it fetches a fresh one in the background.

app.get('/api/products/:id', async (req, res) => {
  const product = await getProduct(req.params.id);

  // Browsers: 60s. CDN: 5 min. Serve stale for 1 min while refreshing.
  res.set(
    'Cache-Control',
    'public, max-age=60, s-maxage=300, stale-while-revalidate=60',
  );
  res.json(product);
});

Layer two: Redis with cache-aside

For data that is shared but too dynamic or too private for the CDN, a Redis cache in front of the database is the workhorse. The most common pattern is cache-aside: read from Redis, on a miss read from the database and write the result back with a TTL.

import Redis from 'ioredis';

const redis = new Redis(process.env.REDIS_URL!);

export async function cached<T>(
  key: string,
  ttlSeconds: number,
  load: () => Promise<T>,
): Promise<T> {
  const hit = await redis.get(key);
  if (hit) return JSON.parse(hit) as T;

  const value = await load();
  await redis.set(key, JSON.stringify(value), 'EX', ttlSeconds);
  return value;
}

// Usage
const product = await cached('product:' + id, 300, () => db.product.findUnique({ where: { id } }));

Choosing TTLs

A TTL is a statement about how stale the business can tolerate a value being. Prices in a checkout flow might tolerate seconds; a list of countries can live for a day. Start by asking the product question, then add a little jitter (a random few percent) to TTLs so that thousands of keys created together do not all expire in the same second.

Stampede protection

A cache stampede happens when a hot key expires and hundreds of concurrent requests all miss at once, all hit the database, and all try to rebuild the same value. Two simple defences work well:

• —Request coalescing inside each process: if a load for a key is already in progress, await the same promise instead of starting another.
• —A short Redis lock: the first request takes a lock with SET key NX EX, rebuilds the value, and others briefly wait or serve the stale copy.

const inflight = new Map<string, Promise<unknown>>();

export function coalesce<T>(key: string, load: () => Promise<T>): Promise<T> {
  const existing = inflight.get(key);
  if (existing) return existing as Promise<T>;

  const p = load().finally(() => inflight.delete(key));
  inflight.set(key, p);
  return p;
}

// cached(key, ttl, () => coalesce(key, loadFromDb))

Invalidation, and what not to cache

The simplest invalidation strategy is a short TTL. When that is not enough, delete the key on write (redis.del after the database commit) or version the key, for example product:42:v7, so a write simply moves readers to a new key. For CDN content, prefer versioned asset URLs over mass invalidations.

Some things should not be cached at all: anything security-sensitive such as permissions checks, balances or inventory at the moment of purchase, and responses so cheap to compute that caching only adds a network hop. A small in-memory cache per instance, with a TTL of a few seconds, is often enough for configuration and feature flags.

Key takeaways

• —Cache in layers: CDN for public responses, Redis for shared data, memory for tiny hot values.
• —Use s-maxage and stale-while-revalidate to let the edge absorb traffic safely.
• —Pick TTLs from business tolerance, add jitter, and protect hot keys from stampedes.
• —Never cache user-specific data in shared caches; invalidate on write or version your keys.

Next up, part 4: Database Scaling: Indexes, Replicas, and Pooling, for the requests that still have to reach the database.