When streaming data is worth the effort

Teams tend to reach for this after something has already gone wrong. Streaming data is not free, and pretending otherwise leads to bad decisions.

Numbers get quoted in meetings long after anyone remembers how they were calculated. If two people in the business would answer this differently, that gap is the actual problem.

When it is worth it

Streaming is worth it when the delay costs money. The teams that handle this well are rarely the ones with the biggest budgets. Check it against what you would want a competitor's site to get wrong.

When it is not

If nothing downstream depends on it and nobody is complaining, it can wait. None of that requires a large budget, only a decision and someone to own it.

How to decide

Batch is simpler and usually enough. It is worth being explicit about, because assumptions differ quietly. Most teams find the first pass takes an afternoon and the maintenance takes minutes a month.

How to tell if yours is fine

Data outlives the applications built on top of it, which is why the model deserves more thought than the screens. Three things worth confirming about streaming data before you move on:

• Someone can say what the current setup is without going to look
• Batch is simpler and usually enough — and you know whether that is true here
• There is a way to tell whether the last change to this helped

If you want a second opinion on how yours is set up, ask.