Culturally, and historically, the number three is very significant. In Christianity there is the Holy Trinity, and in Hinduism, there is the Trimurti. In Chinese, the word for three is considered lucky because it sounds like “alive”, versus four, which sounds like “death”. “Third time’s a charm”, and we count to three when we are about to trigger an event.
Three is also an important number in user research. Three is the number of methods you should use when collecting user research data. It’s based on the idea of triangulation, or sometimes called “mixed method” research. The more methods you use to collect data, the more confident you can be with the conclusions of your research.
Using one user research method gives you one perspective on your target population. That perspective is going to be strongly influenced by the method you use. So you have to be aware of the pros and cons for each research method.
If you run a survey, you are getting a large amount of quantitative (measured) data that is attitudinal (what the user says) in nature. Running an ethnographic study gives you a small amount of qualitative (observational) data that is behavioural (what the user does). I would expect you to mention just one more method here to round it out to 3.
Any research method is subject to biases (measurement, observer, reporting, sampling, etc.) that can lead to incorrect conclusions. You can’t remove all bias from your research methods, but you can be aware of when there is potential bias, and how it influences your results. However, if a similar conclusion is found using different methods, and it is not due to analytic bias, it is more likely to be the truth.
You might believe that doing some research, i.e. a survey, is better than no research. However, if the research is poorly designed, and not validated with other studies, it may be misleading and more detrimental than insightful.
Just like creating a tapestry, interlacing methods that give different perspectives strengthens the findings and creates an image when you step back. Observation-based studies provide a true picture of how the users interact and perform their tasks, while interviews will help your understanding of why they behaved in the observed manner. A survey can measure how common the observed behaviours are in the larger population. Pulled together, you’ll have created detailed descriptions of your user personas, and scenarios of use.
The order that research methods are employed is also important. Collecting behavioural data after you have collected attitudinal data, will help confirm if users were correct when they described their behaviours. Running an attitudinal-based study after collecting observational data allows the user to provide an explanation to their observed behaviour.
Three is also a magic number when evaluating a design and the number of observers in the research. When running a usability study, if three participants make the same error or demonstrate the same behaviour, we call that a trend. For expert reviews, we use three reviewers during the evaluation. Potential usability concerns that are identified by all three reviewers are more likely to be valid user problems, rather than assumed problems.
To ensure that the conclusions from the user research reflect reality, there is no single research method. Using triangulation, by combining different research methods, allows you to balance the weaknesses and strengths of each method. It’s as easy as counting 1, 2, and 3.
Daniel Iaboni, is Senior Experience Architect at Akendi, a firm dedicated to creating intentional experiences through end-to-end experience design. To learn more about Akendi or user experience design, visit www.akendi.com.
Akendi is a product strategy, user experience design and usability research firm. We are passionate about the creation of intentional experiences – whether those involve digital products, physical products, mobile, service or bricks-and-mortar interactions. We work shoulder-to-shoulder to optimize the experiences you deliver. Akendi Corporate Overview (PDF).
Experience Thinking innovation firm in Product UX Strategy, User Experience Design & Usability Testing for Companies: Toronto, Ottawa, Montreal, Vancouver, Canada.
T: +44 (0)20 35982601
22 Highbury Grove
London, N5 2EF