Horizontal Scaling with EC2 Auto Scaling Groups

A bigger server buys you time. More servers buy you headroom and resilience. Horizontal scaling means running several identical instances behind a load balancer and letting the fleet grow and shrink with demand. On AWS, the combination of an Application Load Balancer (ALB) and an EC2 Auto Scaling Group (ASG) is still one of the most dependable ways to do that.

Step one: make the app stateless

Auto scaling only works if any instance can serve any request. That means nothing important can live on the instance itself. Sessions go into Redis or into a signed JWT. Uploaded files go straight to S3. Background jobs go to a queue. Local disk is treated as a scratch space that may vanish at any moment, because with scale-in it will.

• —Sessions: a shared store such as ElastiCache for Redis, or stateless JWTs with short expiry.
• —Files: S3 with pre-signed upload URLs so files never touch the app server.
• —Config: environment variables or SSM Parameter Store, baked into the launch template, never edited by hand on a box.
• —Scheduled jobs: one scheduler (EventBridge, a dedicated worker), not a cron on every instance.

ALB, target groups and health checks

The ALB spreads requests across healthy instances registered in a target group. The health check is the contract between them, so make it meaningful. A good /health endpoint confirms the process can serve traffic: the event loop is responsive and critical dependencies are reachable. Keep it cheap, since it runs every few seconds on every instance.

Launch templates and the scaling policy

A launch template describes exactly how to build an instance: AMI, instance type, security groups, IAM role and user data. Bake your application into the AMI or pull a versioned build at boot; either way, a new instance must become ready without human help.

For the policy itself, prefer target tracking. You pick a metric and a target value, and the ASG adds or removes capacity to hold it there. Average CPU works for compute-bound services. For typical web APIs, ALBRequestCountPerTarget is often better because it scales on the thing that actually changes: traffic per instance.

{
  "TargetValue": 1000,
  "PredefinedMetricSpecification": {
    "PredefinedMetricType": "ALBRequestCountPerTarget",
    "ResourceLabel": "app/my-alb/50dc6c495c0c9188/targetgroup/my-tg/943f017f100becff"
  },
  "DisableScaleIn": false
}

aws autoscaling put-scaling-policy \
  --auto-scaling-group-name api-asg \
  --policy-name requests-per-target \
  --policy-type TargetTrackingScaling \
  --target-tracking-configuration file://target-tracking.json

# Give new instances time to boot before their metrics count
aws autoscaling update-auto-scaling-group \
  --auto-scaling-group-name api-asg \
  --default-instance-warmup 120

Instance warmup matters. Without it, a freshly launched instance reports low load while it is still booting, the average drops, and the ASG may decide it overshot. Set warmup to roughly the time from launch to serving real traffic. Use scale-in protection for instances doing long-running work that must not be interrupted.

Graceful shutdown: scaling in without dropping requests

Scaling out is easy. Scaling in is where users see errors. When the ASG terminates an instance, the ALB first deregisters it and waits for the deregistration delay so in-flight requests can finish. Your process must cooperate: on SIGTERM, stop accepting new connections, finish what is running, close database pools and exit.

import http from 'node:http';
import { app } from './app';
import { db } from './db';

const server = http.createServer(app);
server.keepAliveTimeout = 65_000; // longer than the ALB idle timeout (60s)
server.listen(process.env.PORT ?? 3000);

let shuttingDown = false;

process.on('SIGTERM', () => {
  if (shuttingDown) return;
  shuttingDown = true;
  console.log('SIGTERM received, draining connections');

  server.close(async () => {
    await db.end();
    process.exit(0);
  });

  // Hard stop if draining takes too long
  setTimeout(() => process.exit(1), 25_000).unref();
});

Note the keepAliveTimeout: if Node closes idle keep-alive connections before the ALB does, the ALB can send a request down a socket that is being closed and return a 502. Keeping the server timeout slightly above the load balancer idle timeout avoids that class of intermittent error.

Key takeaways

• —Stateless instances are the prerequisite: sessions, files and jobs live outside the box.
• —Health checks should reflect the ability to serve traffic, not the health of every dependency.
• —Prefer target tracking on requests per target or CPU, and set a realistic instance warmup.
• —Handle SIGTERM, align keep-alive timeouts with the ALB, and let deregistration delay drain requests.

Next up, part 3: Caching Layers: CDN, Redis, and Memory, where we stop doing the same work twice and take load off the whole stack.