Queues and message brokers: what to get right first

It is rarely the thing that gets a project approved, and often the thing that decides how it goes. If you only fix one thing about queues and message brokers this quarter, make it the first item below.

The right architecture for a team of three is the wrong one for a team of thirty, and vice versa. Budget a little time for it every quarter and it never becomes a project of its own.

Start here

Queues absorb spikes that would otherwise take a system down. The reasoning matters more than the rule, because the rule has exceptions. Write the reasoning down alongside the decision, because the reasoning is what changes first.

Then this

Every queue needs a dead letter destination and someone watching it. The cost of getting this wrong is rarely visible on the day it happens. If it only works because one person remembers to do something, it does not work yet.

Eventually

Ordering guarantees are weaker than most teams assume. This is the sort of thing that compounds, quietly, in both directions. Most teams find the first pass takes an afternoon and the maintenance takes minutes a month.

How to tell if yours is fine

Most systems fail at the seams rather than inside any one component. Three things worth confirming about queues and message brokers before you move on:

• Someone can say what the current setup is without going to look
• Queues absorb spikes that would otherwise take a system down — and you know whether that is true here
• There is a way to tell whether the last change to this helped

If you are not sure where your systems currently stand on this, it takes us about an hour to find out.