Posted on: March 17, 2016
Ph.D - Principal Experience Architect
The Power of Three
Is three the magic number?
Culturally, and historically, the number three is very significant. In Christianity there is the Holy Trinity, and in Hinduism, there is the Trimurti. In Chinese, the word for three is considered lucky because it sounds like “alive”, versus four, which sounds like “death”. “Third time’s a charm”, and we count to three when we are about to trigger an event.
Three is also an important number in user research. Three is the number of methods you should use when collecting user research data. It’s based on the idea of triangulation, or sometimes called “mixed method” research. The more methods you use to collect data, the more confident you can be with the conclusions of your research.
Using one user research method gives you one perspective on your target population. That perspective is going to be strongly influenced by the method you use. So you have to be aware of the pros and cons for each research method.
If you run a survey, you are getting a large amount of quantitative (measured) data that is attitudinal (what the user says) in nature. Running an ethnographic study gives you a small amount of qualitative (observational) data that is behavioural (what the user does). I would expect you to mention just one more method here to round it out to 3.
Any research method is subject to biases (measurement, observer, reporting, sampling, etc.) that can lead to incorrect conclusions. You can’t remove all bias from your research methods, but you can be aware of when there is potential bias, and how it influences your results. However, if a similar conclusion is found using different methods, and it is not due to analytic bias, it is more likely to be the truth.
You might believe that doing some research, i.e. a survey, is better than no research. However, if the research is poorly designed, and not validated with other studies, it may be misleading and more detrimental than insightful.
Just like creating a tapestry, interlacing methods that give different perspectives strengthens the findings and creates an image when you step back. Observation-based studies provide a true picture of how the users interact and perform their tasks, while interviews will help your understanding of why they behaved in the observed manner. A survey can measure how common the observed behaviours are in the larger population. Pulled together, you’ll have created detailed descriptions of your user personas, and scenarios of use.
The order that research methods are employed is also important. Collecting behavioural data after you have collected attitudinal data, will help confirm if users were correct when they described their behaviours. Running an attitudinal-based study after collecting observational data allows the user to provide an explanation to their observed behaviour.
Three is also a magic number when evaluating a design and the number of observers in the research. When running a usability study, if three participants make the same error or demonstrate the same behaviour, we call that a trend. For expert reviews, we use three reviewers during the evaluation. Potential usability concerns that are identified by all three reviewers are more likely to be valid user problems, rather than assumed problems.
To ensure that the conclusions from the user research reflect reality, there is no single research method. Using triangulation, by combining different research methods, allows you to balance the weaknesses and strengths of each method. It’s as easy as counting 1, 2, and 3.
Ph.D - Principal Experience Architect
Daniel is dedicated to creating and improving software, website and interface designs to provide an exceptional interaction among users, their environment and their tools. To achieve this goal, Daniel applies knowledge and techniques from engineering, computer science, psychology and anthropology learned while completing a Doctorate in Systems Design Engineering from the University of Waterloo with a focus on human factors, cognitive ergonomics and user-centered design. Prior to joining Akendi, Daniel was the user experience designer for Brock Solutions, and was instrumental in the design of software used by airports and airlines around the world. In addition, Daniel has provided independent human factors consulting services for several large and medium enterprises.
Sign up for our UX Blog
Don't miss an article! We'll notify you of each new post.