How to get device testing right
This is one of those topics that looks small until it costs you something. The short answer to device testing is that it is mostly a sequence of small decisions, not one big one.
You do not control the device, the network, or the operating system version, and the design has to accept that. Anything you cannot measure here, you are deciding by taste, which is fine as long as everyone knows it.
What it costs to ignore
The newest phone is not the phone your customers own. That sounds obvious written down. It is still the thing most often skipped. It is the sort of thing that looks like polish right up until it costs you an enquiry.
The steps
- Establish what you have today before changing anything
- Test on mid-range hardware and poor networks
- Automate the regression path, test the rest by hand
- Write down the decision so the next person does not re-litigate it
Automate the regression path, test the rest by hand. Getting it slightly wrong is survivable. Ignoring it entirely is not. Anything you cannot measure here, you are deciding by taste, which is fine as long as everyone knows it.
Where to go from here
How to tell if yours is fine
The version of the app your customers are running is rarely the one you just shipped. Three things worth confirming about device testing before you move on:
- Someone can say what the current setup is without going to look
- The newest phone is not the phone your customers own — and you know whether that is true here
- There is a way to tell whether the last change to this helped
If any of that sounds like a description of your current setup, it is fixable.