The Art of Asking Questions in UX Research

Surveys, interviews, usability tests and other UX research methods are meant to elicit information to help us clearly understand a user/customer, their needs and their experience when using a service/product; to find the best design/redesign recommendations for a service/product; and in general, to help companies figure out the best possible way to serve their users/customers.

The quality of the information relies entirely on the quality of the questions we ask. A few times I have found myself in a situation where I don’t have enough information to draw clear conclusions about a user’s mental model, needs or experience, and I wish I had asked better or more questions. This has led me to revise and optimize my research instruments so that the next time I am sure I will collect the right information through the right questions.

This is the first article of several to come where I will be talking about how asking the right questions is crucial for success. I will be providing examples from my personal UX research/design practice and tips on how to formulate effective questions for specific methods/techniques and activities that are part of the experience thinking process.

In this first article, I will talk about the fundamentals of asking questions.

“Successful people ask better questions, and as a result, get better answers”
Tony Robbins.

Fundamentals – Open and Closed Questions

Open and closed questions have different goals and it is important to know when it is appropriate to use one type of question over the other.

In qualitative research, we usually look for opportunities to connect closely with a user/customer and open a conversation where we can capture rich information on our topic of interest.

Open questions, as their name says, are good for this purpose, opening opportunities to elicit rich descriptions of what users do/need, and the way they do/need it (e.g. how, when, where, why). Open questions get respondents talking freely, providing as much detail as they want.

Examples:

• “What tools do you use to complete your tasks?”
• “How do you go about completing this task?”
• “Why do you use this tool?”

Closed questions, on the other hand, provide the user/customer with a closed set of implicit or explicit answers without much detail of the context. Closed questions are the way to go for quantitative research with methods such as surveys, where a close connection to the user/customer is not possible (due to large samples) and where the data is usually analysed automatically and in bulk rather than manually and under close scrutiny.

Examples:

• “Do you watch television at home? Yes___ – No___”
• “How would you rate the service provided by ___? a b c”
• “Which one of the following reasons best describes ___? a b c”

In qualitative research, closed questions are also used, but mostly for screening purposes (putting users/customers into categories), to determine whether a certain path in the conversation should be followed, or to close a branch in our conversation by clarifying something said.

Examples:

• “What age range are you in? -Screening.
• “Is having a luxury car important to you? -Screening.
• “Have you visited ___’s website before? [If no, skip to question -Guide]

When used incorrectly, closed questions might be leading or accidentally reveal information that can reduce the validity of the information collected.

Examples:

• “Do you think this icon is good for functionality X?”
  Reveals the purpose of the icon, might lead the user to think that it is obvious.
• “Did you have any difficulties using the ____ tool?”
  Draws attention to a specific feature, which might be unwanted in some cases.
• “Do you think this icon’s colour is inappropriate?”
  The tone of the question might lead the user to think the colour is inappropriate.

For the examples above, better questions would be:

• “What do you think this icon is associated with?”
  Helps reveal the user’s mental model and the effectiveness of the icon.
• “How would you describe using the system?”
  Encourages reflection on the system in general without drawing attention to a specific feature or aspect. If the answer doesn’t address the level of difficulty and there’s a reason to get a firm yes/no, then follow with: “Did you have any difficulties using the system in general?”
• “How do you find the icons?”
  Doesn’t draw attention to a specific icon or aspect. If specific feedback on a certain aspect of the icons is needed, follow with another open and possibly a closed question, e.g.: “What do you think about their colour?”, “Do you think this icon’s colour is appropriate or not appropriate?”

Probes and Funnelling

Sometimes, (open and closed) questions require probes for clarification, completeness, and drilling down for more details. Probe questions might be open or closed, depending on how much more detail is required.

Example:

• Main question: “How do you perform this task?”
• Probe 1: “Where do you look for that type of information?”
• Probe 2: “Are there any other ways you would want to accomplish the task?”
• Probe 3: “What are your thoughts as you go through this process?”

Probes are very common in several UX research techniques. They are particularly useful in usability testing to get the user talking about their thought processes as they interact with the system.

Sometimes, asking “why?” or “can you elaborate on that?” after the respondent answers the main question is a simple yet powerful way of getting rich and detailed information that we can use to have a clear and complete context of the topic in question.

A common technique for eliciting detailed information on a process or task is Funnel Questioning. In this technique, an open question invites the user to describe the task/process under study, multiple subsequent probes are used to fill necessary details, and a final summary helps confirm that the information captured is correct and complete.

Example:

• “Can you tell me how you went about?”‚ – Open
• “What motivated you to start?” – Open
• “What did you have in mind during this stage?” – Probe
• “What did you do next? – Probe
• “Who helped you during that stage?” – Probe
• “How did you keep track of …” – Probe
• “How much time did you put into” Closed – clarifying
• “How many times did you do that?” Closed – clarifying
• “Did you do anything else?” – confirming
• “Let me see if I understood correctly”‚ Summary

This technique is particularly useful when eliciting information for experience mapping.

“The important thing is not to stop questioning. Curiosity has its own reason for existing”
Albert Einstein

Final Remarks and What is Next

In this first article, I talked about the fundamentals of using open and closed questions to elicit quality information necessary in UX research and design.

With this basic understanding of questions in general, we can design better instruments to help us conduct a better Experience Thinking process.

–

In the next article, I will talk about how the right questions help us conduct effective usability testing sessions and effective interviews to help us develop User/Customer Personas and Experience Maps.

Got anything you would like to add or ask about? Do you have any personal experience where you think the right questions helped you or could have helped you do UX better? Please share below!