Serverless at Scale: When AWS Lambda Fits

Lambda's promise is that you never think about servers. At a million requests you still think about capacity, just in different units: concurrency, cold starts and downstream connections. Understand those three and Lambda is a superb tool for spiky, event-driven work. Ignore them and it throttles at exactly the wrong moment.

Concurrency is your real capacity

Each concurrent request runs in its own execution environment. Required concurrency is roughly requests per second × average duration: 2,000 requests per second at 200 ms needs about 400 concurrent executions. The default quota is 1,000 concurrent executions per account per Region, shared by every function in that account. You can raise it through Service Quotas, but request the increase well before launch day.

Lambda also limits how quickly a function can scale up, so an instant jump from zero to a huge burst can still see throttles (429 TooManyRequestsException) while new environments start. Queue-driven workloads absorb this naturally; synchronous APIs need a plan.

Reserved and provisioned concurrency

• —Reserved concurrency guarantees a function a slice of the account quota and also caps it. Use it so a noisy background function cannot starve your API, and to protect a fragile database behind a function.
• —Provisioned concurrency keeps a number of environments initialised and warm. It removes cold starts for that capacity, and you pay for it whether or not it is used.

# Cap and guarantee concurrency for the API function
aws lambda put-function-concurrency \
  --function-name orders-api \
  --reserved-concurrent-executions 300

# Keep 50 warm environments on the "live" alias
aws lambda put-provisioned-concurrency-config \
  --function-name orders-api \
  --qualifier live \
  --provisioned-concurrent-executions 50

Taming cold starts

A cold start is the time to create an environment, load your code and run everything outside the handler. On a busy function, only a small fraction of requests hit one, but those requests land squarely in your p99.

• —Ship less code. Bundle and tree-shake with esbuild, and import only the AWS SDK v3 clients you use.
• —Initialise outside the handler. SDK clients, config and connection pools are created once per environment and reused across invocations.
• —Give it memory. CPU scales with memory, so more memory often means faster init and a lower overall bill.
• —Use provisioned concurrency for latency-critical paths, and SnapStart on runtimes that support it.

Databases: the hidden bottleneck

Lambda scales out far faster than a relational database can accept connections. A thousand concurrent environments each opening its own Postgres connection will exhaust max_connections long before Lambda runs out of capacity. Put RDS Proxy in between: it pools and reuses connections, so thousands of functions share a small, stable set.

import { Pool } from 'pg';
import type { APIGatewayProxyEventV2 } from 'aws-lambda';

// Created once per execution environment, not per request
const pool = new Pool({
  host: process.env.DB_PROXY_ENDPOINT, // RDS Proxy, not the instance
  max: 1, // one connection per environment
  idleTimeoutMillis: 60_000,
});

export const handler = async (event: APIGatewayProxyEventV2) => {
  const id = event.pathParameters?.id;
  const { rows } = await pool.query(
    'SELECT id, status, total FROM orders WHERE id = $1',
    [id],
  );
  return rows[0]
    ? { statusCode: 200, body: JSON.stringify(rows[0]) }
    : { statusCode: 404, body: '{"error":"not_found"}' };
};

When Lambda is not the answer

Lambda bills per request and per millisecond of compute. For spiky, idle-heavy or event-driven traffic, that is hard to beat. For steady, high, predictable traffic running around the clock, a handful of containers on ECS or Fargate, or EC2 in an Auto Scaling group, is usually cheaper and gives you long-lived connections and in-process caches for free.

• —Great fit: webhooks, queue consumers, scheduled jobs, image processing, APIs with unpredictable bursts.
• —Think twice: constant high-throughput APIs, WebSocket-heavy apps, and latency-critical paths where cold starts matter.
• —Often best: a hybrid, with containers for the core API and Lambda for the event-driven edges around it.

Key takeaways

• —Concurrency is requests per second times duration. Know your quota and raise it early.
• —Use reserved concurrency to protect neighbours and provisioned concurrency to kill cold starts.
• —Initialise clients outside the handler, and put RDS Proxy in front of relational databases.
• —Compare costs honestly: steady traffic often belongs on containers.

You now have a lot of moving parts. The only way to know they hold together is to break them on purpose. Next: Load Testing Before Launch with k6.