When device testing is worth the effort

There is no clever trick in this one, just a handful of decisions worth making deliberately. Device testing is not free, and pretending otherwise leads to bad decisions.

You do not control the device, the network, or the operating system version, and the design has to accept that. If two people in the business would answer this differently, that gap is the actual problem.

When it is worth it

The newest phone is not the phone your customers own. That sounds obvious written down. It is still the thing most often skipped. Check it against what you would want a competitor's site to get wrong.

When it is not

If nothing downstream depends on it and nobody is complaining, it can wait. There is a version of this that is over-engineered, and it is worth avoiding.

How to decide

Automate the regression path, test the rest by hand. There is a version of this that is over-engineered, and it is worth avoiding. Most teams find the first pass takes an afternoon and the maintenance takes minutes a month.

In practice

Mobile releases are slower to correct than web ones, so the cost of shipping a mistake is higher. Three things worth confirming about device testing before you move on:

• Someone can say what the current setup is without going to look
• The newest phone is not the phone your customers own — and you know whether that is true here
• There is a way to tell whether the last change to this helped

The point is not perfection, it is knowing which of these you have consciously chosen to skip.