The Two Biggest Issues Threatening The UX Industry Today

I’m compelled to comment on two recent experiences as I see them as detrimental to the UX industry.

Serious Confusion About UX Processes

Friends, let me tell you a story.

Our story begins with an RFP from the government asking for “Focus Group Usability Testing”.

That’s not a real thing.

The fact that about 50 different people would have written, edited, vetted, approved, posted that RFP and not one of them knew, or maybe cared that Focus Group Usability Testing is not a real thing is worrying to me.

Another worrying thing is that UX companies actually responded to the RFP and said “sure, we can do that for you.”

I feel that it’s not entirely the fault of the government agency requesting the service, nor entirely the fault of the UX service providers, instead I consider it a perfect storm of miscommunication, misinterpretation and eagerness on both sides.

Here is a bit of history:  In 2010 a citizen sued the federal government because she was unable to apply for a job online.

The suit cited a lack of accessibility and usability as the major barriers to being able to complete the application process. In response to losing this court case, the government instituted the standard on web accessibility, the standard on web interoperability and the standard on web usability.

Section 6.1.5 of the standard on web usability states:  Ensuring that the Web development process meets the user-centred design requirements described in Appendix F.

Appendix F states:  The following requirements ensure that users can effectively and efficiently find, understand and use the information and services provided through websites and Web applications. 

Web managers, functional specialists, Web content owners and equivalents are responsible for ensuring that, for each website or Web application:

1. User research methods are used to define
   • The users of the website or Web application
   • The tasks the users will accomplish using the website or Web application
   • The contexts in which the users will use the website or Web application to accomplish their tasks.
2. The users, their tasks and the contexts in which the users accomplish their tasks form the basis for the design of the website or Web application.
3. The website or Web application is designed iteratively; the design process starts with low-fidelity methods, such as mock-ups, and progressively moves to higher-fidelity methods, such as prototypes.
4. The design of the website or Web application is regularly evaluated and improved through usability testing, such as observing users completing tasks, throughout its life cycle.
5. The design of the website or Web application takes into consideration the other service channels, such as telephone and in-person, used to obtain information and services from the Government of Canada.

While this is a loose definition of usability testing, I see the advantage in it, as it opens the door to a variety of legitimate usability inspection methods. However, the requirement is clear.  The acceptable methodologies require the observation of users completing tasks.

That doesn’t happen in a focus group.  The loose definition also opens the door to all kinds of odd bastardizations that do not preserve internal or external validity and would at best produce questionable data.

The whole idea behind usability testing is that it is an experiment conducted in a controlled environment.  You are controlling for extraneous variables so that the change in the independent variable can be attributed to a change in the dependent variable with a high degree of certainty,  i.e. your design changes, are what makes the application/website/service easier to use, and not some other factor like learning or better hardware.

Confusion Around The Value Of UX Services

The other part of this problem is one that our industry shares with our clients: Nobody has a clear idea of how much these services are worth.

This leads to service providers offering the cheapest possible inspection methods because they are afraid of being priced out of running.

The Design Value Index has shown that design focused companies have consistently outperformed the companies listed on the S&P 500 by over 200%…  think about it.

While this isn’t the most empirical of metrics, we can still conclude that a focus on good, evidence based design works.  The way to gather that evidence is not by doing “focus group usability testing”.

The money spent on marketing bad designs, building bad designs, supporting bad designs dwarfs the money people are willing to spend to ensure that the design of their products support user needs, and are effective, efficient and satisfying to use.

The fact that there are charlatans in the industry that will promise 3 rounds of usability testing plus IA and interaction design for under 25K further compounds the problem.

What Is The True Value Of What We Do?

A couple of years ago a small client of ours was acquired by a larger company.

During the due diligence prior to acquisition, auditors from the buyer were reviewing the company’s financials and noticed a sharp upswing in revenue.

When the auditors enquired about what change precipitated the upward trend, they were told it was the point at which Akendi was brought in to redesign the product’s UX.

UX works.

Good UX, executed by professionals, is worth the investment.

The facilities, knowledge, planning, management and methods that are necessary to conduct proper usability inspection are worth paying for.