Posted on: 10 February 2015
5 Things to Consider When Conducting Usability Testing on Mobile Devices
When it comes to repeatedly creating exceptional user experiences, there are few tools in the UX practitioner’s toolbox as critical as usability testing. As mobile has become such a crucial domain for User Experience, mobile usability testing has played a critical role in shaping the usability of mobile experiences. In this article I will discuss some of the lessons I have learned in conducting mobile usability testing that may help you.
Covering the Basics
In basic terms, usability testing is a method of determining if users can use your software or tool to achieve a given task. To do this, we bring real users into a lab, put them in front of the system and ask them to complete a series of tasks. We observe where they run into issues and adjust our designs to make the product easier to use. With mobile testing, all of the basic components are the same, with the exception of a the following considerations:
Mobile Usability Test Equipment
Figure 1: mobile usability testing setup
First of all, when I teach our Usability Testing course, I have participants conduct a usability test without any special equipment or software. The most important piece of equipment in any usability test is the researcher. With a clipboard and a protocol, any UX practitioner worth their salt should be able to conduct a usability test no matter what the platform. That said, there are several tools out there to help support data capture during a mobile usability test.
Although screen-capture works well on desktop, you can miss a lot when testing a touch device. On a desktop, when a user clicks on an element on screen that is not clickable, the screen capture will pick up the cursor clicking on that area. As there is no cursor on touch devices, these users actions can be missed if you only use screen capture. Using a camera to capture user actions can be much more informative.
Keeping a camera focused on a mobile device while someone is using it can be difficult though. Products like Mr. Tappy can be helpful in doing this without disrupting the user’s experience. Mr. Tappy is a camera arm that attaches to the back of mobile devices and holds a camera over the screen (see figure 1).
When using a device like this, I suggest
- Shortening the arm as much as possible to reduce camera vibration
- Turn off auto focus on the camera so it doesn’t focus on the user’s finger instead of the screen
- Have an additional camera pointed at the user’s face to capture their reactions to the system
- Ensure the camera arm is well secured to the back of the device
- Use an external microphone to ensure moving the device does not disrupt the audio recording
Context of Use
The biggest challenge for mobile usability is context of use. When designing mobile applications or websites, the context of the user, whether they are driving a car, sitting on a bus or in their living room at home, can have a huge impact on their performance with the system.
Users won’t be using your mobile application or website in an environment that resembles a usability lab. At the same time, tying to run a usability test on a bus or while someone is driving a car will make it impossible to maintain consistency between sessions. It may also make it difficult to reliably record your session or ensure the safety of your participants.
The solution here is simple: usability test in the lab for usability issues and use other research to identify contextual issues. Specifically contextual inquiry, ethnographic research, and user interviews can help uncover contextual factors that may impact the usability of your product.
Mobile Software Platforms
Although, deciding which platforms to design for is a topic for another blog, it’s important to discuss which platform(s) you should test on. This decision can be particularly difficult if you are testing a responsive website, which could be viewed on any mobile device.
In the end, you should try to match your testing to the usage rates of your product. If the product you are testing is already in market and you know that half your participants are iOS users and half are Blackberry users, then it makes sense to test half your sample on iPhones and half on Blackberry’s. If you are testing something that is not yet in market, try to make an educated guess to what you expect that proportion will be based on competitors in the market or the current market share of your target demographic.
Regardless of the breakdown of devices you decide to test, make sure you have at least 3 to 4 users testing on each device and that each participant is testing on the device they have experience using. If you test iPhone users on an Android device, you will be testing the usability of the device rather than your product.
If you are early in the design process, consider testing a prototype to avoid picking a specific platform altogether. Paper prototypes also have the benefit of being quick and easy to produce and allow you to test early in the process when changes are cheap and design alternatives are high.
Even with these considerations, a mobile usability test is still just a usability test. Focusing on the basics with these considerations in mind will ensure your test is successful and your final product is usable.