Why device testing matters more than it looks

The advice here is unglamorous, which is probably why it gets skipped. Device testing is easy to treat as a detail, and that is exactly why it is worth a few minutes of attention.

Mobile releases are slower to correct than web ones, so the cost of shipping a mistake is higher. It is the sort of thing that looks like polish right up until it costs you an enquiry.

Why it matters

The newest phone is not the phone your customers own. In practice this is a scheduling problem more than a technical one. The version that survives contact with a real deadline is the simple one.

Test on mid-range hardware and poor networks. This is the sort of thing that compounds, quietly, in both directions. It rarely shows up as a line item, which is exactly why it slips.

Warning signs

Automate the regression path, test the rest by hand. This is the sort of thing that compounds, quietly, in both directions. If it only works because one person remembers to do something, it does not work yet.

The short version

The version of the app your customers are running is rarely the one you just shipped. Three things worth confirming about device testing before you move on:

• Someone can say what the current setup is without going to look
• Automate the regression path, test the rest by hand — and you know whether that is true here
• There is a way to tell whether the last change to this helped

If you want a second opinion on how yours is set up, ask.