A queue sits between the part of a system that creates work (the producer) and the part that does it (the consumers or workers). The producer drops a message on the queue and moves on. Workers pull messages off and process them at their own pace. Queues let you absorb traffic spikes without dropping requests, and let you scale producers and consumers separately.
How it works#
Producers add messages at one end and workers take them from the other.
- A message can be anything: a short string, a JSON job description, an event.
- The queue stores messages durably until a worker acknowledges it has finished one.
- Producers and consumers don't know about each other. You can restart, deploy or scale the workers with little impact on producers.
Smoothing load. Say a spike of 1,000 requests arrives but your workers can process 200 per second. Without a queue, 800 requests fail. With a queue, they wait a few seconds and then get processed. Nothing is dropped.
Two common uses#
Most queues in a design do one of two jobs.
- Buffer bursty traffic. A ride-sharing app sees huge spikes at rush hour or after a concert. Ride requests go on a queue, and the matching service works through them at a steady rate instead of falling over.
- Distribute work. A photo service puts each "apply this filter" job on a queue. A pool of workers pulls jobs, so the expensive image processing is spread evenly across machines.
What you need to know#
Five concepts come up whenever you use a queue.
| Concept | What it means |
|---|---|
| Ordering | Many queues are roughly first in, first out. Kafka guarantees order only within a partition, so related messages must share a partition key. Some managed queues offer strict FIFO as an option with lower throughput. |
| Retries | If a worker fails or doesn't acknowledge in time, the message is delivered again. You can usually configure the number of attempts and the delay between them. Because of retries, workers should be idempotent: processing a message twice must be safe. |
| Dead letter queue (DLQ) | After too many failed attempts, a message moves to a separate queue. You can inspect it, fix the bug and replay it, instead of having it block or vanish. |
| Partitions | A queue can be split across servers to handle more messages. Each partition is consumed by its own worker or set of workers. As with database sharding, you choose a partition key so related messages land together. |
| Backpressure | A way to slow producers down when consumers can't keep up. See below. |
Backpressure: the queue can hide a capacity problem#
Queues make it easy to overload a system without noticing.
Picture a kitchen that can cook 200 meals an hour while orders arrive at 300 an hour. If you keep accepting every order, the pile of pending orders grows forever.
Wait times climb until the whole thing collapses. The queue didn't fix the problem. It hid it.
Backpressure pushes back on the producer when you're overloaded:
- When the queue is full, reject new requests with an error such as
429 Too Many Requests, or - Slow down how fast you accept new messages.
The producer now knows to back off, so the problem is visible and handled at the source.
Without backpressure:
- The producer sends 300/s.
- The queue grows without limit.
- Latency explodes or the system crashes.
With backpressure:
- The producer sends 300/s.
- The queue fills up.
- 100/s are rejected.
- The producer slows down.
- The system is stable at 200/s.
A highway on-ramp light is a good mental model. When the highway is congested, the light stops cars from entering. Drivers wait a bit, but the highway keeps moving.
Example. A pipeline processes thousands of events per second and calls a slow downstream service (say, a machine learning model). If that service slows down and nothing pushes back, the queue balloons with work that can't be finished in time. A good design watches queue depth and either throttles ingestion or rejects upstream producers before the backlog becomes unrecoverable.
Trade-offs and pitfalls#
Queues add their own costs.
- Latency: a queue adds delay. If a request has a tight latency budget (for example, under 500 ms end to end), putting a queue in the synchronous path will almost certainly break it. Use queues for work the user doesn't wait on.
- Duplicates: most queues deliver at least once. Design consumers so a repeated message doesn't charge a card twice.
- Hidden backlog: monitor queue depth and message age, and alert when they grow.
- Ordering vs throughput: strict global order limits you to one consumer. Ordering per key (per user, per order) scales much better.
Common technologies#
A few systems cover most use cases.
- Apache Kafka: a distributed log often used as a high-throughput queue or event stream.
- Amazon SQS: a fully managed queue on AWS, with standard and FIFO variants.
- RabbitMQ and Google Cloud Pub/Sub are other common choices.
Key takeaways#
- A queue decouples producers from consumers so each can scale and fail independently.
- Queues absorb bursts: extra work waits instead of being dropped.
- Use retries plus a dead letter queue for failures, and make consumers idempotent.
- Partition keys give you scale while keeping related messages in order.
- A queue can hide a capacity shortfall. Use backpressure and watch queue depth.