Always Be Testing

Usability is defined as the effectiveness, efficiency and satisfaction with which specified users achieve specified goals in particular environments [1].

I recently watched the movie Glengarry Glen Ross (I know, I’m about 20 years late) and although it’s an old movie, the famous message conveyed “Always Be Closing” still holds true. In the field of web and product design, there’s a similar mantra that we should all follow: Always Be Testing.

Due to the level of competition that now exists between online services, good usability is becoming increasingly important. If a user encounters errors or can’t find what they are looking for on your website, they will likely move on to the next. So even if a site looks great and has lots of features if a user can’t figure out how to use it, they will move on.

The only way to ensure good usability (no matter how good a designer you are) is by conduct testing. However, it’s not always clear how to run an effective study. With so many tools & techniques available it can often be overwhelming, so to help provide some guidance, I’ve included 3 valuable yet under-utilized evaluation techniques to help get you started:

Card Based Classification Evaluation

Card-based classification evaluation, more commonly known as tree testing is a simple technique for evaluating if users are able to find specific information on a website. A large website is typically organized into a hierarchy (a “tree”) of topics and subtopics. Tree testing provides a way to measure how well users can find items in this hierarchy.

The example below shows the stages of a tree test where a user was asked to identify where they would find a wireless mouse within the hierarchy of information:

Step 1 -� Select a top-level category

Step 2 -� Chose a sub-category

Step 3 -� Select the 3rd level grouping that would contain a wireless mouse

Step 4 -� Confirm selection

The results of a tree test not only help to identify if a user can find specific information, but it’s also valuable to note how much time it took them to complete the task and if they made errors along the way. Each data point will help to uncover potential issues.

Rapid Prototype Testing

One of the common excuses that I hear for not conducting usability testing is that the website is not ready, not in a testable state. That’s where prototyping can help.

Rapid prototyping is the process of quickly mocking up the future state of a website and validating it with a representative set of users. Doing this rapidly and iteratively generates feedback early and often in the process, improving the final design and reducing the need for changes during development.

Prototypes can be created using low-fidelity techniques such as paper and pencil sketches or they can be created using more high-fidelity tools such as Illustrator or Photoshop. There are also some great purpose build prototyping tools out there such as Omnigraffle, Axure and InVision.  Ultimately the tool that you use is not that important, the key is to choose one that you’re comfortable using and one that will allow you to make changes quickly based on the results of your tests.

A/B Testing

Web analytics provides data about how people are using your website, which is a valuable insight. However the best data-driven companies don’t just passively store and analyze data, they actively generate actionable data by running experiments.

An A/B test is an experiment where you present 2 different versions of a web page to 2 sets of random users. Differences in usage patterns between the two versions help identify what works best. The diagram below shows how different design variations of a website can lead to significantly different conversion rates.

The benefit of A/B testing is that you can run it on your live site, eliminating the need to recruit test participants and run a formal in-person study.

Many research techniques available to you as a researcher. Testing early and often is a vital step in ensuring good usability, so next time you’re designing a product or adding a new feature just remember to always be testing.

[1] ISO 9241 definition