Investing in UX Research: Increasing the Quality of Requirements

In my UX consulting life I go through many, sometimes brief, cycles of meeting a potential new client, talk about what their pain points are, what we offer and how we can help address their pain points. In the course of these sales journeys I see some conversations come up more than others.

The pain we hear most often from our potential clients is that there is an urgent need to redesign the user experience of a desktop and mobile app or web portal. Most of the time it needs to be designed so it behaves responsively, combined with the fact that the potential client has already figured out their strategy. This means the client thinks they know what their end-user/customer needs. They have a document with high level and sometimes highly detailed requirements intermixed (often labelled a web strategy or business requirements), and they’d like Akendi to take it from there, i.e. can we provide a quote to create the experience?

Building the Right Thing

So what’s the issue with this picture? Seems a fairly straightforward design job and many design teams work this way; they receive requirements from marketing, product management, executives or other stakeholders and off they go. The challenge is that unless we know what the process our potential client followed to arrive at these requirements, (AND we learn that their process included a set of decent customer and user involvement activities) we cannot trust enough the validity of these requirements. The number of assumptions not backed by customer or user evidence in the requirements can wreak havoc on the product success in the marketplace. At this stage in the game we cannot say to ourselves confidently that we are building the right thing.

Now that’s a troubling realization and one that potentially doesn’t go over very well with a client that hasn’t signed the contract yet. The way we convey this message is usually with some honest questions: ‘What are your user groups? Can you tell me the top 5-10 tasks with your app? Do you know, really know, when and where the product is used?’ We can ask these questions at this point as we’re not engaged yet and can easily ask for forgiveness in our initial ignorance of the product and company. The responses vary but very rarely do we get a clear answer to all three questions.

Customers and Users

The first question however, ‘Who is your user?’ is usually answered with the most information. We hear stories of who buys the product, what analytics say and how product management sees the market. The conversation moves to the distinction between who buys the product (customer) and who uses it (user), an important distinction that results in product requirements that are often not captured separately.  The first one, customer experience, needs research and design that focusses primarily on identifying the desired emotional response and value proposition understanding. The second one, user experience, puts more focus on use: make the transaction, find the information, getting things done, in a much more utilitarian way. The research and design is more behavioural based and ‘logical’ if you will.

Tasks and Context of Use

Teasing out these differences between customers and users often leads to insightful conversations around what users and customers really do with the product, site or mobile app. The second question, what are the top 5-10 tasks users do on the product, site, app?, is then easier in a way as our potential client starts to see the holes in their understanding and foundation for their current requirements. Again the analytics are brought up often here as a way to show that it’s known what people do on the site until the question about why people are going to these high frequency pages can’t be answered confidently. To get to an answer as to what users are trying  to achieve with the product moves the conversation to using techniques like contextual inquiry, ethnographic observation and task analysis to uncover what users really do with the content and functionality.

Project Scope Creep

It turns out to be a difficult point in the overall sales conversation as the potential client sees the initial scope of the project grow. They thought they were buying design and now we’re starting to revisit product requirements and challenge the scope of what needs to be built. Also, completely new elements are put into the mix with research techniques that first need an explanation of what the technique entails before we can talk about the value and need for them. This last point is amplified somewhat when I talk about the research that needs to happen to understand when and where the product is used. Similar research techniques as before will result in added scope and budget.

So what to do? Sometimes the urgency to deliver results is so great that the client stops here and asks us to first get going with design. For me, this is where I need to make an educated business decision if I’d want to go ahead with some form of engagement that doesn’t involve any experience research. It’s a really tough one as the absence of research is not only a bad practice but touches on one of the core pillars of user centered design (together with design iteration and experience testing). I’d guesstimate that one in six projects that we do would be done without much research, the others have various degrees of insight before we jump into design and most of them have a form of experience testing after the concept designs and high fidelity designs are done. It’s a key part of our offering.

The Business Impact of no Experience Research

There is a big catch to this I find for us as a UX firm: a high correlation between not doing research and having project time/money overruns, less happy internal UX design teams and more often less happy clients. Almost every time I succumb to the ‘design without research’ projects we end up churning more, are a bit unhappy about the results ourselves in some way and together with the client we end up in a place where we say to each other; what if we had done this project with research?  This is because we always, always end up talking during the project about the (big) assumptions we’re making about what the user needs at various stages of the workflow; how we have no way of knowing unless we ask users in their place of use and that the types of user groups we imagine may be distributed differently and thus greatly influence what functions and content would bubble to the top. In these projects we rely heavily on the client contact who is often a product manager, marketing manager or director of engineering. We make them increasingly uncomfortable with our questions to help steer the design and unfortunately we end up behaving more like a contractor.

The Quality of Requirements

I think there is an established best practice here. The current quality of most requirements leave something to be desired and with a known approach that works, it’s agonizing sometime to not see it happen all the time -“ why wouldn’t you do this? is something I ask some of our potential clients in these cases. The business benefit is clear. Awareness and education of our clients leads increasingly to the situation that for the next project they do reserve time and budget for the experience research phase, even in an Agile development environment. They realize that ultimately the quality of their product requirements depend on it.