A usability test is the cheapest way to learn what data cannot tell you. It does not reveal how many people have a problem—other tools do that. It does reveal exactly what the problem is, at a speed no analysis can match. Five half-hour conversations are usually enough to uncover what the team has missed for months.
How many people you really need
Qualitative research follows a different logic from statistics. You are not looking for a representative sample; you are looking for recurring problems. In practice, you begin to hear the same comments by the third participant, and the fifth rarely contributes anything entirely new within a single test script.
Scroll the table sideways to see all columns.
| Situation | Number of people | Note |
|---|---|---|
| One scenario, one customer group | Five people | A standard starting point, sufficient for initial conclusions |
| Two clearly different customer groups | Three or four from each group | The groups usually get lost in different places |
| Checking a fix | Three people | Enough to see whether the problem has disappeared |
| Asking which version people prefer | A usability test cannot answer this | It is a quantitative question, not a qualitative one |
The final row matters. Usability tests answer ‘can people do this, and where do they get lost?’ rather than ‘which version sells better?’. Answering the second question takes traffic and patience—we cover it in the article about website A/B testing.
Five people will tell you what the problem is. Recordings and maps will tell you how many customers it affects. Explore behaviour analysis in Meetlead.
Whom to invite and where to find them
The most common excuse is ‘we have nowhere to find participants’. In practice, you need only follow two rules: the person must not work for your company and should have real experience of the problem you solve.
- Real customersThe best source. Many will agree in return for a small gift or simply out of goodwill.
- People in a similar situationIf you sell to repair shops, ask somebody who runs one, not an office colleague.
- Friends outside the industryWorse than customers, much better than nobody. They will spot problems with navigation and language.
- Whom not to inviteTeam members, your agency or anyone who knows the website by heart.
Be direct and honest: ‘We are testing our website, not you. We want to find the places where something is unclear.’ That sentence changes the participant’s mindset and produces more of the critical comments you need.
The script: a task, not a tour
The most common beginner’s mistake is to ask somebody to ‘look around the website and say what you think’. You will get a design review. A test works when the participant has a specific task and a goal of their own.
Scroll the table sideways to see all columns.
| Instead of | Say | Why |
|---|---|---|
| ‘Rate this website’ | ‘Find out how much implementation would cost for a company of your size’ | A task reveals a path; an opinion reveals taste |
| ‘Click Pricing’ | ‘Find out whether this solution fits your budget’ | Do not name the element they need to find |
| ‘Fill in the form’ | ‘Schedule a call for next week’ | Use the customer’s goal, not the feature name |
| ‘Is this clear?’ | ‘What will you do next?’ | A closed question almost always gets an affirmative answer |
Prepare two tasks, or three at most, for each session. You will not have time for more, and a tired participant stops commenting. Arrange tasks in the customer’s natural order: understand the offer first, then find the price, and finally make contact or buy.
The instruction must describe the participant’s goal, never an interface element. If a task names the button, the test has just lost its purpose.
The simplest version of this research is a five-second test that checks only the first impression. We describe it in the article about the first screen of a website.
How to facilitate a session without distorting the result
This is the hardest part, and the easiest to describe. Your job is to stay quiet for much longer than feels comfortable. Every hint destroys the observation you came to collect.
- Explain the rules at the beginning.We are testing the website, not the person. There are no wrong answers. Please think aloud.
- Ask them to comment aloud.Asking ‘What are you looking for now?’ once at the beginning is enough to get the participant talking.
- Do not prompt, even when it hurts.Thirty seconds of silence is a valuable observation, not an awkward pause.
- Turn questions back to the participant.When they ask ‘Where is it?’, answer ‘Where would you look for it?’.
- Record behaviour, not opinions.‘She returned to the menu three times’ is a fact. ‘She disliked the layout’ is an interpretation.
- Ask follow-up questions only at the end.After the tasks, return to moments of hesitation and ask what was happening then.
Separating fact from interpretation requires the same discipline as working with recordings—we describe it in the article about analysing session recordings. A test is easier because you can ask questions, but harder because it is easy to suggest the answer.
A usability test or session recordings?
These two tools are sometimes treated as substitutes, but they answer different questions and work best together. Recordings offer scale and natural behaviour; a test lets you ask follow-up questions.
Scroll the table sideways to see all columns.
| Criterion | Usability test | Session recordings |
|---|---|---|
| Participant motivation | Artificial, created by the task | Genuine, because the person has their own goal |
| Ability to ask follow-up questions | Yes—this is the main advantage | No—you are left to interpret |
| Scale | A few people | All traffic |
| Time to insight | One day | Depends on how the material is selected |
| Best use | Understand what the problem is | See how often it occurs |
The sensible order depends on what you already know. If the data shows a drop and you do not understand it, start with recordings and run a test on that specific step. If you are building something new and do not have traffic yet, a test is your only available source of evidence.
After making a change, use recordings and maps to check whether the problem has disappeared for real users. Schedule implementation and we will help set up that measurement.
What to do with the results
- Write down observations immediately after the session, before they fade—ideally during it.
- Group them by location on the website, not by participant.
- Mark problems that occurred for at least three people.
- Separate defects you can fix immediately from hypotheses that require a decision.
- Choose one change and make it before planning the next round.
- Repeat the test with three people after the change to see whether the problem has disappeared.
Test results fit neatly into the wider process of improving a website: observation, hypothesis, priority, change and verification. We describe the complete cycle in the website conversion audit, and explain how to combine it with quantitative data in the article about what Google Analytics cannot show you.
One final note: recording a test session that includes the participant requires their consent, and the material contains their image and voice. This is governed differently from recording website traffic and should be handled separately.
Before three repeated test observations become a project, verify them across all traffic. Explore behaviour analysis.
If an observation reveals a bug, hand over reproduction steps and the expected acceptance result. Use the customer bug report handoff template so evidence stays available when work moves between teams.
Scroll the preview sideways or enlarge it to see the details.
