Device testing: what to get right first

It is rarely the thing that gets a project approved, and often the thing that decides how it goes. If you only fix one thing about device testing this quarter, make it the first item below.

The version of the app your customers are running is rarely the one you just shipped. The failure mode is not doing it wrong, it is doing it once and assuming it stays done.

Start here

The newest phone is not the phone your customers own. There is a version of this that is over-engineered, and it is worth avoiding. The failure mode is not doing it wrong, it is doing it once and assuming it stays done.

Then this

Test on mid-range hardware and poor networks. Getting it slightly wrong is survivable. Ignoring it entirely is not. The failure mode is not doing it wrong, it is doing it once and assuming it stays done.

Eventually

Automate the regression path, test the rest by hand. The cost of getting this wrong is rarely visible on the day it happens. It is worth deciding this deliberately rather than inheriting whatever the last person set up.

What this looks like day to day

Mobile releases are slower to correct than web ones, so the cost of shipping a mistake is higher. Three things worth confirming about device testing before you move on:

• Someone can say what the current setup is without going to look
• Test on mid-range hardware and poor networks — and you know whether that is true here
• There is a way to tell whether the last change to this helped

Worth checking on your own setup before it becomes someone else's problem to fix.