The world has truly gone mobile. At up to 70 percent, mobile web traffic now significantly surpasses desktop, and the number of smartphone users worldwide — most of them in the U.S., China, and India — is estimated at more than three billion (according to Statista). Its forecast to further grow by several hundred million in the next few years. Whether it’s emerging markets that are catching up fast, or the younger generation in the western world, for a lot of people the smartphone is now the primary device to access the internet. In the U.S. alone, consumers even spent a staggering 90 percent of their mobile time in apps.
This means that user testing apps has become absolutely crucial for optimizing the UX and discoverability of digital products and services. Given the importance of mobile, you should include a mobile usability test in your research, which can provide you with invaluable data of how people actually use your app in the real world.
To get you started, we quizzed four leading experts on mobile app design and the essential factors in planning and conducting a mobile user test. Read on and discover the techniques that, in their experience, lead to better digital products.
Test your apps early and often
Chris How, principal UX consultant at strategic design and innovation agency Clearleft, has found that the maxim of ‘test early and often’ really holds true.
While How recommends involving your intended audience at every stage of the process (from initial concepts, through lo-fi prototypes, and to the production-ready product), lead UX/UI designer and mobile expert Stéphanie Walter also suggests pre-testing before inviting real users.
“There’s nothing more frustrating than gathering the wrong data of false positives or negatives because there was an issue with the test protocol, the prototype didn’t send the user to the right page, or the vocabulary you used in your task description biased the user,” she explains. “You can avoid all that with pre-tests, which are really quick and simple to set up.”
A pre-test conducted with colleagues before testing the mobile prototype with real users.Just grab some colleagues and ask them to play the role of the user. The rest should be pretty much the same as it would be for official participants.
“If the test is supposed to happen in a particular room with a particular device, use the same environment and setting for the pre-test,” Walter advises. “Go through the welcome speech, the tasks, and the mobile prototype, like you would do for the real test. If there’s an issue in the prototype, such as a typo, or a loop in your protocol, you should be able to detect it during these ‘dry run’ sessions. If there’s some language that might bias users, or the way you ask questions isn’t quite clear, you can detect this upfront, too.”
According to Walter, pre-tests are a little bit like a wedding rehearsal: they help you make sure that everything runs smoothly for the big usability testing day.
Reduce bias and consider context
Femke van Schoonhoven, a product designer at Uber, points out that, to ensure a successful user test, it’s important to reduce bias and consider context when creating your prototype.
“It’s good to remember that you are not your user, and whoever you’re testing with will be seeing your designs for the first time, without any of the context you have.”
Before you run your next test, van Schoonhoven therefore recommends making sure you’ve done the following:
- Hide your hotspots to encourage organic discovery
- Use real content instead of stock imagery and Lorem Ipsum
- Tweak the prototype to cater to the specific user (for example, use a female profile in the design if the tester is female
- Localize your content if necessary
- Encourage realness by testing on the intended device
- Always link one level deeper than the test to allow the user to explore (for example, let Help go to Help even though it’s not part of your test)
Considering the above will help validate your test and help your participant view your prototype through a realistic lens, which will avoid distractions and keep the test organic.
Recreate a natural environment
For Andy Vitale, strategic design and innovation leader and member of Adobe’s Design Circle, the key to usability testing mobile apps is to try to recreate the natural mobile environment as closely as possible throughout the various fidelities of the designs being evaluated.
“I have seen the greatest successes in mobile testing sessions when people are allowed to test directly on mobile devices, so that they can be observed using native mobile gestures and behaviors,” Vitale explains. “From a research perspective, it’s not about asking people what they would like from the design but more about observing them and the way they interact with the design and learning how to better help them achieve their desired results. It’s about testing a hypothesis.”
Vitale recommends understanding and aligning as a team on which success metrics you are trying to measure with the designs. If there is also a way to capture quantitative data throughout testing, it will help you set baselines and show improvements throughout various iterations of the designs you will be testing.
Meanwhile, Ida Aalen, chief product officer and co-founder of video conferencing startup Confrere and proponent of easy and affordable user testing techniques, has learned that it’s never about the equipment.
“The main thing is to make sure you’re not asking leading questions, in addition to encouraging users to think out loud as they try using your app,” she advises.
As your view of the screen will often be obscured by a participant’s hands, Clearleft’s principal UX consultant Chris How also views it as vital to actively listen and watch users for small clues indicating delight or frustration. He therefore recommends really getting to know the flow of your own app before conducting a mobile usability test.
Test your app in the wild
If you can, you should also get out of the lab and test the app where it’ll be used.
“Usage and behavior are linked to context and environment,” How explains. “When testing a mobile app for a museum, I shadowed participants. Seeing how often their phone went in and out of their pocket helped focus the design on supporting short bursts of repeated use. Likewise, testing a travel app in the wild highlighted the need for a design that could be seen in bright sunlight and used whilst wearing gloves on freezing days.”
Accessibility in particular should now be considered central to creating products that are fit for purpose in a mobile-first world. Whether you’re using your smartphone while you’re walking along, possibly one-handed, a coffee in the other hand, are distracted or in a rush — you’re temporarily impaired and in need of inclusive design. Conducting a mobile user test while you’re out and about will throw up exactly these issues and help you understand the unique UX requirements you need to consider for your app.
You will need to carefully adapt and plan just how you carry out your tests to meet the needs of your particular project. There’s no denying, however, that user testing apps — early and often, with colleagues as well as real users, on a variety of devices, in the lab and in the wild — is the key to the success of your app. Get it right, and users will come back to it again and again.