How to Conduct Usability Testing: Research Methods and Approaches

Why carry out research

We have already written about the website redesign process. Here we look in more detail at usability testing research methods, through which you gather the data needed to design a solution and test the functionality of the design. Based on the test results, you refine the design and test again.

Unfortunately, we cannot offer a step⁠-⁠by⁠-⁠step recipe for usability testing. It always depends on:

• what you need to find out,
• the nature and complexity of the solution being tested,
• we evaluate the data and translate the results into a design solution,
• and last but not least, your time and financial constraints.

Below, however, we describe the basic user research methods that you can use for almost any digital product, from a presentational website to e⁠-⁠commerce to mobile apps.

Moderated usability testing and its advantages

First and foremost, we recommend conducting moderated usability testing. The respondent navigates through the website according to a prepared scenario that reflects the expected journey of a website visitor. They complete individual tasks and comment on their steps.

It is called moderated testing because the respondent is guided through the test by a facilitator, who is either present in person or joins remotely. They continually follow up with the respondent without leading them towards a particular answer. At the same time, they partially evaluate the responses and non⁠-⁠verbal communication in real time.

Prepare the usability testing scenario with the desired website goals and input data in mind. Gather input data from web analytics, heat maps, or an expert analysis in which a UX designer reviews and evaluates the website. It often pays to consult the scenario directly with the client for whom the website is being created. This helps uncover critical or potentially problematic areas of the website, which you can then focus on during testing.

A problematic area might be, for example, that visitors browse the e⁠-⁠shop but do not add items to the cart. Focus on this action during testing and ask the respondent to purchase a specific item. During this process you might discover they have trouble finding the add⁠-⁠to⁠-⁠cart button, or that they are missing information important for completing the purchase. During testing you can ask them where they expect the button to be, or what specific information they feel is missing.

The advantage of moderated testing is not only the creation of hypotheses for subsequent quantitative verification, but above all finding solutions. From the respondent you find out what is unclear during their navigation of the website, what information is missing, or what obstacles they encounter. You learn how they think about what they want to do on the website, what they expect to happen when they click on specific elements. This allows you to see the real problem and understand what needs to be addressed.

There are usually two approaches to moderated usability testing.

Krug testing

The first approach is called Krug testing (after Steve Krug, a well⁠-⁠known advocate of website usability testing). Using his method, you verify that visitors understand the designed website and can easily reach their goal. It involves testing the complete prototype, refining it based on results, and optionally retesting. Depending on the number of iterations, this method can be time⁠-⁠consuming and costly, so it tends to be used for larger companies and corporations where research is separate from the actual design work.

For the appropriate number of respondents, we follow the findings of Nielsen Norman Group. Their research found that informative results are obtained when testing with five people from a specific target group — a group that needs to accomplish a similar thing on the website and has similar motivations.

For example, if you have an e⁠-⁠shop selling bicycles, you might want to test with people who have never bought a bike before, and also with competitive cyclists. These two groups will approach the website in fundamentally different ways, so they cannot be tested together as one group.

RITE method testing

For smaller projects, you are more likely to use the RITE method — Rapid Iterative Testing and Evaluation. This is essentially the same as Krug testing but more iterative. You select six respondents and split them into three iterations. In the first phase, only two people test the website; you immediately refine the design based on results, and the next two respondents test it.

Even after the first round, you catch the most significant issues and fix them. The downside can be lower result accuracy — a problem is tested with only two people. If more respondents tested it, the original issue might turn out to be less relevant.

Usability testing is best complemented by in⁠-⁠depth interviews, where you talk to respondents about the service in general. Testing then focuses on the navigation through the e⁠-⁠shop.

One round of testing with one person typically takes 45 to 90 minutes, depending on the scope and any in⁠-⁠depth interview.

Evaluating UX testing results

For both methods, testing involves a facilitator and at least one other team member. Their job is to take notes and offer an additional perspective on results. With the respondent's consent, you can also make a recording to refer back to later.

Based on the testing data, begin the analysis. For organising findings, you can use tools such as Miro or a board with sticky notes. It is a demanding and time⁠-⁠consuming process, because qualitative testing data is softer and less clear⁠-⁠cut. But it leads you to specific problems and the root causes behind them. Look for solutions to the problems identified as most important.

Beyond providing input for designs, usability testing and in⁠-⁠depth interview results serve as hypotheses that can be further tested in quantitative research. If five people encounter a problem on the website, you can see the problem exists. But you do not yet know whether it only affects a handful of people. Quantitative testing with more respondents will tell you whether it is truly a priority to solve.

Unmoderated usability testing

You can complement moderated usability testing with additional testing to verify the functionality of individual design decisions. Unmoderated testing is sufficient here — it allows you to gather answers from more people more easily and quickly, resulting in statistically more meaningful data.

Unmoderated methods include five⁠-⁠second tests for assessing first impressions, checking credibility, and evaluating the user's initial orientation. The respondent is shown a design for five seconds. You then find out what they remember and whether they found key information at first glance.

Preference and emotional tests

Preference testing involves showing respondents two versions of a website design and asking them to choose the one that suits them better. A potential issue is result bias — respondents select what they think is better, but you cannot confirm it actually is. For more detailed information, you will need moderated usability testing.

This method is particularly useful for emotional testing of graphics and visuals, asking how a particular design makes the visitor feel (more playful, professional, trustworthy, etc.). If you give the respondent space in an open question to explain their choice, such testing becomes even more valuable.

Another form of emotional testing is Product Reaction Cards. The respondent sees the website design and receives cards with adjectives — positive, negative and neutral. There are 118 cards in total, though typically only a subset is used (we usually work with 60). The respondent selects cards that best describe what they see, helping you verify whether the proposed design aligns with the required brand identity.

In addition to the website design or prototype, also test the information architecture using the Card Sorting method. The respondent receives a list of products and sorts them into groups, which they may also name. Based on this, you choose how to categorise products, or verify that their mental model matches yours. Similarly, you can sort individual pages and sub⁠-⁠pages.

Next may come the tree testing method, where the respondent navigates through the website structure to find, for example, a specific product page. This verifies that the proposed structure matches how your users actually search for products.

Usability testing is the foundation of every redesign

There is a wide range of testing methods that can be combined and adapted to each project's needs. The number and type of tests depends on the project, its current stage, where it needs to go, the budget, and deadlines. Usability testing — at minimum website usability testing — is something we recommend in every case.

Every limitation of research methods leads to less reliable results, which feeds through into the design. The design may then not perform as well as it could have with more thorough research. The more you invest in research, the lower the risk the website will not reach its full potential. And basic testing can be managed even on a smaller budget.