Posted on: 17 March 2016
Ph.D - Chief Experience Officer
The Power of Three
Is three the magic number?
Culturally, and historically, the number three is very significant. In Christianity, there is the Holy Trinity, and in Hinduism, there is the Trimurti. In Chinese, the word for three is considered lucky because it sounds like “alive”, versus four, which sounds like “death”. “Third time’s a charm”, and we count to three when we are about to trigger an event.
Three is also an important number in user research. Three is the number of methods you should use when collecting user research data. It’s based on the idea of triangulation or sometimes called “mixed method” research. The more methods you use to collect data, the more confident you can be with the conclusions of your research.
Using one user research method gives you one perspective on your target population. That perspective is going to be strongly influenced by the method you use. So you have to be aware of the pros and cons of each research method.
If you run a survey, you are getting a large amount of quantitative (measured) data that is attitudinal (what the user says) in nature. Running an ethnographic study gives you a small amount of qualitative (observational) data that is behavioural (what the user does). I would expect you to mention just one more method here to round it out to 3.
Any research method is subject to biases (measurement, observer, reporting, sampling, etc.) that can lead to incorrect conclusions. You can’t remove all bias from your research methods, but you can be aware of when there is a potential bias, and how it influences your results. However, if a similar conclusion is found using different methods, and it is not due to analytic bias, it is more likely to be the truth.
You might believe that doing some research, i.e. a survey is better than no research. However, if the research is poorly designed, and not validated with other studies, it may be misleading and more detrimental than insightful.
Just like creating a tapestry, interlacing methods that give different perspectives strengthens the findings and creates an image when you step back. Observation-based studies provide a true picture of how the users interact and perform their tasks, while interviews will help your understanding of why they behaved in the observed manner. A survey can measure how common the observed behaviours are in the larger population. Pulled together, you’ll have created detailed descriptions of your user personas, and scenarios of use.
The order that research methods are employed is also important. Collecting behavioural data after you have collected attitudinal data, will help confirm if users were correct when they described their behaviours. Running an attitudinal-based study after collecting observational data allows the user to provide an explanation of their observed behaviour.
Three is also a magic number when evaluating a design and the number of observers in the research. When running a usability study, if three participants make the same error or demonstrate the same behaviour, we call that a trend. For expert reviews, we use three reviewers during the evaluation. Potential usability concerns that are identified by all three reviewers are more likely to be valid user problems, rather than assumed problems.
To ensure that the conclusions from the user research reflect reality, there is no single research method. Using triangulation, by combining different research methods, allows you to balance the weaknesses and strengths of each method. It’s as easy as counting 1, 2, and 3.
Ph.D - Chief Experience Officer
Dan firmly believes that technology must be created with the user in mind. Never shy to critique a bad design, Dan uses the Akendi blog to shine a spotlight on usability mistakes…and their solutions. Leveraging his background in engineering, computer science, psychology, and anthropology, Dan offers a unique perspective on the latest UX trends and techniques.