Streaming data for small teams

Most teams know this matters. Fewer have decided who owns it. Most advice about streaming data assumes a team that does not exist at your size. Here is the version that does not.

Data outlives the applications built on top of it, which is why the model deserves more thought than the screens. It is worth deciding this deliberately rather than inheriting whatever the last person set up.

What to keep

Streaming is worth it when the delay costs money. This is the sort of thing that compounds, quietly, in both directions. The teams that stay on top of it are the ones who put it on a calendar rather than a wish list.

What to drop

Process that exists to coordinate ten people is overhead when there are two of you. The teams that handle this well are rarely the ones with the biggest budgets.

How we handle it

Batch is simpler and usually enough. None of that requires a large budget, only a decision and someone to own it. If two people in the business would answer this differently, that gap is the actual problem.

How to tell if yours is fine

Numbers get quoted in meetings long after anyone remembers how they were calculated. Three things worth confirming about streaming data before you move on:

• Someone can say what the current setup is without going to look
• Streaming is worth it when the delay costs money — and you know whether that is true here
• There is a way to tell whether the last change to this helped

If any of that sounds like a description of your current setup, it is fixable.