An API gateway is the single front door to a set of backend services. Clients send every request to the gateway, and it forwards each one to the right service, for example GET /users/123 to the users service. Along the way it handles the chores every service would otherwise repeat: checking who the caller is, limiting how fast they can call, and logging what happened. In a microservice architecture it is usually the first box a client request touches.
What a gateway does#
A gateway takes on eight jobs that every service would otherwise repeat.
| Job | What it means |
|---|---|
| Single entry point | Clients know one URL instead of ten service addresses. The gateway routes internally. |
| Routing | Maps paths and methods (/users/*, /orders/*) to the service that owns them. |
| Authentication and authorization | Validates JWTs, API keys or OAuth tokens once, before traffic reaches any service. |
| Rate limiting and throttling | Caps requests per user, per IP or per endpoint to stop abuse and control cost. |
| Request and response transformation | Adds or strips headers, reshapes payloads, or translates between protocols. |
| Caching | Returns stored responses for repeated identical requests so the backend isn't hit. |
| Observability | One place for logs, metrics and traces across all services. |
| TLS termination | Decrypts HTTPS at the edge and forwards plain HTTP inside the private network. |
Without a gateway, every service has to implement auth, rate limiting and logging on its own. That is duplicated code and duplicated bugs. The gateway becomes the contract between the outside world and your internal system.
Gateway vs load balancer#
The two are often confused because both sit in front of servers and forward traffic. They solve different problems.
| Load balancer | API gateway | |
|---|---|---|
| Main job | Spread traffic across healthy instances of one service | Decide which service a request belongs to, and apply policies |
| Understands | Connections, and for L7 balancers, basic HTTP | HTTP semantics: routes, headers, tokens, bodies |
| Typical features | Health checks, failover, balancing algorithms | Auth, rate limits, transformation, caching, API keys |
| Overhead | Low, built for raw throughput | Higher, because it does more work per request |
A common layout uses both. The gateway applies policies and picks the service, then a load balancer spreads that service's traffic across its instances.
A request passes through four hops in order:
- Client
- API gateway
- Load balancer
- Service instances
On AWS, a typical version is:
| Hop | Typical AWS service |
|---|---|
| API gateway | Amazon API Gateway |
| Load balancer | Application Load Balancer |
| Service instances | ECS or EC2 instances |
A popular serverless variant is API Gateway in front of AWS Lambda. The gateway handles auth, throttling and routing, so each function contains only business logic.
When to use which#
Use both when:
- A service runs on several instances that need traffic spread across them (load balancer's job), and
- You also need auth, rate limiting or routing before requests reach those instances (gateway's job).
- At high scale, the gateway does the smart work once and the load balancer handles raw throughput.
One is enough when:
- Small app, one service: a load balancer alone is fine.
- Serverless functions: a gateway alone works, because the platform scales the functions for you.
- Internal service-to-service traffic: usually a load balancer or a service mesh, without a gateway.
Common gateways#
Several products fill this role.
- Amazon API Gateway (managed)
- Kong (open source and commercial)
- Apigee (Google Cloud)
- A plain nginx or Apache HTTP Server fleet can also play this role. Large companies have historically used big web-server fleets as their gateway layer.
Trade-offs and pitfalls#
A gateway is useful, but it is not free.
- Extra hop: every request pays a small latency cost for passing through the gateway.
- Single point of failure: run several gateway instances behind a load balancer or use a managed service.
- Logic creep: keep business logic out of the gateway. It should route and enforce policy, not decide prices or approve orders.
- In design interviews, adding a gateway as the client's first contact is a sensible default, but interviewers rarely dig into it. Spend your time on the parts specific to the problem.
Key takeaways#
- An API gateway is the single entry point that routes requests to services and handles cross-cutting concerns like auth, rate limiting and logging.
- A load balancer spreads traffic across instances of one service. A gateway decides which service and applies policy.
- Many systems use both: gateway first, then a load balancer per service.
- Serverless setups often need only a gateway. Small single-service apps often need only a load balancer.