Five mistakes teams make with observability
The advice here is unglamorous, which is probably why it gets skipped. These are the ones we run into repeatedly when we audit observability.
Operability is a feature, and it has to be built rather than bought. Assume whoever inherits this will have half your context and none of your patience.
The mistakes we see most
- Treating it as a launch task rather than an ongoing one
- Assuming someone else already owns it
- Monitoring tells you something broke, observability tells you why
- Traces, metrics, and logs answer different questions
- Never checking whether the fix actually worked
Instrument the paths that lose money first. There is a version of this that is over-engineered, and it is worth avoiding. Check it against what you would want a competitor's site to get wrong.
What to do next
What this looks like day to day
The bill is a design document: it tells you exactly what your architecture actually does. Three things worth confirming about observability before you move on:
- Someone can say what the current setup is without going to look
- Monitoring tells you something broke, observability tells you why — and you know whether that is true here
- There is a way to tell whether the last change to this helped
If you are not sure where your systems currently stand on this, it takes us about an hour to find out.