Key takeaways from the article:
- Always consider what you want to research, who you want to research it with, and whether you have access to those respondents.
- Usability testing can be done in a pinch with randomly selected respondents, but if you're testing website content and context, you need a specific target audience.
- You can invite respondents to your office, connect online, or conduct testing at their homes — it depends on the conditions you need for testing.
Moderated testing: one method, adaptable to any project
Whether you use moderated testing to evaluate the usability of a performance website, an e-shop, or for emotional testing of visual design — whether you're verifying the function of a single button or testing an entire e-shop flow — the core testing method is always the same. You walk a respondent through a prepared scenario, let them complete tasks, and ask follow-up questions for details.
Shape this universal method according to what each project or client needs.
Consider:
- testing goals – do you need to deeply understand how people behave across several complex tasks, or are you focused on the basic usability of the most important pages,
- scope and complexity of the tested solution – is it a single-page booking site or a large e-shop,
- target audiences – do you have one well-defined target group you can access, or several very different groups that are also hard to reach,
- your time constraints,
- budget.
Only then can you make the right decisions about:
- how detailed your scenario will be – whether it will include a few short tasks or a comprehensive walk-through of the entire website,
- how precisely you'll target your respondent recruitment and where you'll find them,
- how many respondents you'll need
(to have a representative sample)<divider><a href="https://www.nngroup.com/articles/how-many-test-users/" target="_blank">https://www.nngroup.com/articles/how-many-test-users/</a> – Nielsen Norman Group go into more detail about everything that can play a role., - what exactly the output of the testing should look like and how to use it going forward.
„Always assess whether moderated testing will yield enough information to have added value and be worth your time."
Selecting respondents is the hardest part of testing
Once you know what you want to test and how, you're faced with arguably the hardest decision — who you'll test with. When selecting respondents, always consider:
- the testing goals — what you want to verify,
- whether you need a specific target audience or whether people from your network will do,
- where you'll find them and whether you can actually reach them.
As you think it through, focus on the cost-effectiveness ratio. If you need to find out whether visitors understand the basic layout of a page and can identify clickable elements, a perfectly targeted audience will give you precise data — but you'll reach similar conclusions with people from your network, for far less time and money. On the other hand, if you're testing a complex website flow aimed at domain experts, testing with people from your network will be a waste of time, because they'll approach all the content in a completely different way than your actual target audience.
„If you want to know whether a user can navigate the website regardless of content, a quicker and simpler approach will do. But if the topic is more complex, or the test scenario focuses on understanding customers' motivations, fears, and expectations, you need to carefully select respondents from the client's target audience."
The quick approach is suitable for testing the usability of smaller elements and simple tasks. The target audience doesn't matter — you can test with people from your network. Consider using colleagues (who aren't working on the project) or randomly recruited respondents. One extra tip: when selecting respondents, it's good to mix people with varying levels of digital literacy and different levels of web experience. This helps you verify that your solution is understandable and usable by everyone.
The in-depth approach is used for usability testing in the context of website content. Therefore, choose respondents from the target audience who know the context. Reaching them may be more difficult, which is why this approach is generally more expensive and time-consuming. For testing the usability of large and specialized solutions, however, it is absolutely essential and provides accurate data on user motivations, concerns, and needs.
3 ways to conduct moderated UX testing
Once you've selected your respondents, think about how the actual testing session will run.
Moderated user testing relies on the presence of a facilitator, who guides the respondent through the scenario and asks follow-up questions. Ideally, another person observes and takes notes (or the session is recorded). That's the essence of moderated testing. But decide whether the testing will take place:
- in person at your offices or another formal setting,
- remotely,
- or at the respondent's home.
Choose the most suitable and effective approach based on the testing goal and what you need to find out.
To simulate a real-life situation as closely as possible, test directly at the respondent's location. Keep in mind, though, that traveling to respondents takes time and money. It's definitely worth it when using the website extends into the user's own environment — and it's therefore essential to observe them in that context during testing.
For example, when we tested an application for managing a metering point for E.ON Distribuce with users a few years ago. In the app, users were filling out forms and needed to look up a contract or check the electricity meter. In an office setting, the client would have simply commented on the steps — and we would never have discovered that they kept walking several floors to locate the meter. Testing at respondents' homes gave us a much better picture of what using the app actually involves.
A more cost-effective option is remote testing, where the respondent stays in their own environment and you connect via a video call. This approach also works well for verifying simple functionalities where traveling for just a few minutes of testing would be excessive.
However, it can be more difficult to ensure the respondent understands all your instructions remotely. Technical issues can also disrupt the session and are harder to resolve when you're not sitting next to the participant or can't offer them a different device.
For more complex testing scenarios, also consider whether important details — such as non-verbal communication and body language — might be missed during remote testing. Always insist on a video call to minimize distortion as much as possible.
If you're conducting more extensive testing with a complex scenario or need greater interaction with the respondent than a screen allows, opt for in-office testing. This approach is often the most effective. At the same time, it creates what is essentially a lab environment, where the respondent may behave differently than they would at home or in a real-life situation.
„If you can't test real-life situations, ask about the respondent's last or most significant relevant experience. For example: When you last filled out a form, what did you have to do — what documents did you look up, what made navigating the website difficult?"
One-time or iterative testing evaluation
The most common methods are the Krug approach and its more agile variant, RITE. We use the Krug approach when we need to test the entire website flow comprehensively. RITE, on the other hand, is used when we need to iterate more quickly and gradually refine a specific part of the flow. This also determines how findings are evaluated and implemented.
With the Krug approach, you first conduct testing with 5 people from the target audience, as recommended by NN Group. You evaluate all the data together and only then implement the findings in one go. During testing, record all insights — but don't evaluate anything yet, so the facilitator isn't tempted to focus on certain things and inadvertent bias doesn't creep in. The findings can also serve as a foundation for quantitative research (e.g., a survey).
With the RITE approach, testing is typically conducted with 6 respondents, in separate rounds of 2. Evaluate the findings from the first round immediately and implement them into the solution before testing again in the second round. The interim results thus serve as hypotheses for the design, which you then verify in subsequent rounds.
Whatever approach you choose, always consider it carefully so that moderated user testing delivers the most accurate results possible and provides the greatest added value.
