How to design a scalable API gateway

What an API Gateway Is

An API gateway is the single entry point for all client traffic. It handles cross-cutting concerns so individual services don't have to.

Core Responsibilities

• Authentication: validate JWT or API keys before forwarding requests
• Rate Limiting: enforce per-user or per-IP request limits
• Routing: forward requests to the correct downstream service
• Protocol Translation: accept REST from clients, forward gRPC to services
• Response Aggregation: combine multiple service responses into one

Rate Limiting at the Gateway

# Nginx rate limiting config example
limit_req_zone $binary_remote_addr zone=api:10m rate=100r/m;

server {
  location /api/ {
    limit_req zone=api burst=20 nodelay;
    proxy_pass http://backend;
  }
}

Gateway vs Load Balancer

A load balancer distributes traffic. A gateway handles business logic like auth and rate limits. In production, you typically have both: a load balancer in front of multiple gateway instances.

Popular choices: AWS API Gateway, Kong, Nginx, Envoy. In interviews, mention the gateway pattern and name one or two concrete implementations.