Making the Urgent Case for User Research

A common ask from clients and students is for me to help them explain the reasons for conducting user research within their organization. While it’d be great to provide robust cost-to-benefit ratio data or ROI metrics, getting such data isn’t realistic. It would also be based on assumptions. I prefer to make the argument that any research, is better than none-“which is the current state for most organizations.

First, to construct the argument for research let’s say you need to decide on the design of a product or service. In this case, you have an initial idea of how to proceed. That idea or knowledge is assumed (as it hasn’t been validated by data) and is either correct or incorrect. You might have confidence in the reliability of that information, but that confidence may be over or underestimated due to biases. There’s a 50/50 chance of being right or wrong.

Next, you have the option of conducting research. The consequence here is that you either switch or don’t switch from your initial assumed knowledge.

The model for all possible outcomes given these options is summarized in the table below.

Without Research

In the case where you don’t conduct research, we eliminate the switch possibility (because you would never switch from your initial position without any external influence unless you accept that you are an irrational person. 

This leaves the scenario where you don’t conduct any research and your initial assumed knowledge is correct, resulting in a correct guess. Alternatively, your initial assumed knowledge is incorrect, and the lack of research means that you are heading down the wrong path. So, you have about a 50/50 chance for a desirable outcome.

With Research

If you conduct some research, assuming the amount of research is fitting to the scale of the research question, then you either may end up either switching or remaining with your initial assumed knowledge. 

If you conduct research and switch, but your initial knowledge is correct, then you are going to make faulty decisions based on improper research.

Improper research includes:

• Talking to the wrong people
• Asking the wrong question(s)
• Using the wrong research methods
• Bias in the analysis

As a researcher, it is your responsibility to minimize the probability of this scenario. For this analysis, let’s assume that this scenario is as likely as the other research scenarios.

If you switch based on research, and your initial assumptions were incorrect, then you can course-correct and increase your chances of success.

If you conduct research and find your initial assumptions are correct, then the value of the research is validated. You can then proceed with confidence knowing that you are correct. 

Finally, if your research shows that your initial assumptions are incorrect, and you don’t switch then you are practicing negligence and increasing risk.

Looking at these scenarios, in the worst case, you should have only a 25% chance of unknowingly making an error after researching versus a 50% chance with no research.  As mentioned earlier, that 25% can be further reduced by correctly planning your research.

50%

This reasoning shows that doing any research reduces the risk of an unfavourable outcome by 50% versus conducting no research. This increases as you reduce the probability of bad research through planning and proficiency. 

And will your next initiative include research?