Most queue diagrams look the same: one producer, one queue, several workers. The point is to share work out. More workers, more throughput.
Then one day you write a system where the same event goes into two different queues, each with its own consumer, and someone asks why you didn’t just use one.
The answer is that those two queues aren’t doing the same job. This post explains the fan-out pattern, what load leveling means, and why it is not load balancing.
The pattern you already know
Classic producer-consumer looks like this:
1 | producer ──► queue ──► worker 1 |
Three workers, all identical, all doing the same thing. A job goes to whichever one is free.
The goal here is load balancing — spreading work across workers so no single one is swamped. Add a fourth worker and you go faster. This is the competing consumers pattern, and it’s the right default.
But notice the assumption baked in: every job is interchangeable. Any worker can take any job. That’s what makes balancing meaningful.
The pattern when jobs aren’t interchangeable
Now say every API request produces one event. That event feeds two things:
A dashboard — how many requests, how fast, how many errors
A billing ledger — what to charge the customer
Same event. Completely different requirements.
The dashboard wants speed and doesn’t mind if a row goes missing. The billing ledger must never lose a row, ever, and would rather be slow than wrong.
Put both behind one queue and you’ve married them. The strict, careful, retry-everything billing writer sets the pace for the dashboard. Your graphs go stale because an invoice row is being retried.
So you fan out:
1 | API server |
One producer writes the event twice, into two independent queues. Each queue has its own consumer with its own rules. They land in the same store at the end, but they get there separately.
Nothing is being balanced here. Nobody is sharing work. The two branches do different things.
So what is load leveling?
Load leveling means putting a queue between a fast producer and a slow consumer so that bursts get absorbed instead of dropped. The queue holds the spike. The consumer drains it at its own comfortable pace. Load balancing spreads work across workers. Load leveling spreads work across time.
That’s the whole idea. A coffee shop analogy:
Load balancing = opening a second till so two lines move at once.
Load leveling = handing customers a buzzer. The kitchen keeps cooking at its normal speed, and nobody is stuck at the counter.
The kitchen never got faster. It just stopped being the thing everyone waits on.
Microsoft’s architecture guide calls this queue-based load leveling, and it’s worth reading. The core promise: your producer can spike to 10× for thirty seconds and your consumer never notices, because the queue ate the spike.
Why two queues instead of one
Here’s the part that trips people up. You could put both consumers on a single queue. Why not?
Because a queue levels load for whatever is behind it. One queue means one buffer, one retry policy, one backlog, one failure. Two queues means two.
| dashboard branch | billing branch | |
|---|---|---|
| Wants | fresh data | correct data |
| Losing a row | annoying | unacceptable |
| Retries | give up fast | retry hard |
| If it backs up | graphs lag | invoices lag |
Different answers in every row. That’s the tell. When two consumers of the same event disagree about what “good” means, they need their own queues.
If they shared one, the strictest policy would win by default and quietly become everyone’s policy.
There’s also a blunt failure argument. One queue, one consumer down, everything stops. Two queues, billing down, the dashboard is still live — and you can replay the billing backlog later, because it’s still sitting in its own queue, untouched.
The rule of thumb
Ask one question: are these consumers doing the same job, or different jobs?
Same job (three workers rendering thumbnails) → one queue, many workers. Load balancing. Add workers to go faster.
Different jobs (one bills, one graphs) → one queue each. Fan-out plus load leveling. Adding workers to one branch does nothing for the other.
They stack, by the way. Each branch of a fan-out can have competing consumers inside it. Fan out for independence, balance within a branch for throughput. This is just publish-subscribe on the outside and competing consumers on the inside.
When not to do this
Fan-out isn’t free, and it’s easy to over-apply.
Two consumers, same requirements? Use one queue. You’ve added a moving part for nothing.
You need both branches to agree instantly? Fan-out is asynchronous by design. The two branches drift, briefly. If that drift is unacceptable, you want a transaction, not queues.
The producer can now half-fail. It might write to queue A and fail on queue B. Decide up front whether that’s tolerable, or whether you need the write to be atomic.
That last one is the real cost. Before, you had one write to get wrong. Now you have two.
Takeaway
Load balancing and load leveling sound similar and solve opposite problems.
Load balancing answers “this is too much work for one worker.” Add workers.
Load leveling answers “this arrives faster than it can be handled.” Add a queue and let time absorb it.
Fan-out is what you reach for when one event feeds several consumers that disagree about durability, freshness, or retries. Give each one its own queue, and their bad days stop being each other’s bad days.
If you want to reason about how deep those queues actually get, Little’s Law is the tool — it ties queue length, arrival rate, and wait time together in one line of arithmetic.
And the dashboard branch above has a mirror-image problem worth knowing about. Queues exist because the producer is faster than the consumer, so you buffer. Metrics collection is the opposite: nothing is buffered anywhere, and the number does not exist until someone asks for it. Scraping a machine that has no open ports walks through that side.
Suggested image: side-by-side illustration of the two coffee-shop scenarios — a second till opening (balancing) versus customers holding buzzers (leveling). Alt text: “Load balancing adds a second till; load leveling hands out buzzers.”
