Wherever You Go: Testing Mobile Applications, Part 1

I took over testing for a mobile application several years ago, and they had focused solely on functional testing (here’s a requirement, here's what it's supposed to do; here’s what we're saying it does; we've tested these things in the lab, and they seem to work), and they were failing. One of the first things I did was to look at the device and find all the options. Most of us just get it set up the way we like it and then we don't go back in—who wants to mess up your own device? So, on a new test device, I will spend several hours going into every nook and cranny to find these things that are built in and how they affect things. One of the fascinating ones was "reverse high contrast" used as an accessibility setting. So, instead of white with black text, you could flip it to black with white text. Trying that, the application that I was testing didn't work at all.

I created a mnemonic after doing a lot of this kind of testing called "I SLICED UP FUN," where I look at combining these activities with other things—inputs, store location, interruptions and interactions, communication, ergonomics, data, usability, platforms, functional tests, user scenarios, and networking. I started going through user scenarios (this is an enterprise-level app) to find out what people are going to use. Well, how do I use my device? I take notes on it, I put in calendar events, I get a call, I blow the call off, I text people—all kinds of things. I started using the built-in parts of the device in conjunction with our application: Use the application, go check email, send a text, look at a calendar event. While I'm using the application, it's slow to load, so I'm bored. I switch context and check my email, then I check back to see if the data file has loaded.

It was shocking to find out how these interruptions and interactions with these built-in things especially could cause our application to crash or to freeze up or to get weird error messages. We found that with one particular device type with one particular application we were testing, if we saved new calendar events or recorded voice messages with the built-in recorder app or saved notes with the built-in note-taking application prior to using our application, the first time it tried to connect to the Internet, it would blow up. Of course, we can't control these other applications, but we were able to put in better error messaging and do something so the application could recover.

That was something that took me back years and years with PCs, where something else going on in the environment or on the device could cause problems. Certainly, incorporating these things that take precedence—these built-in things, these settings—into the testing scenarios that you're using for your actual application is incredibly important.

Read "Wherever You Go: Testing Mobile Applications, Part 2"