Your chances of developing successful products grow when you make usability testing a part of the project. That is if it’s done without any mistakes.
I’ve used various testing methods, including in-person, remote, moderated, and unmoderated testing, to ensure that our team delivers the right product.
Yet, carrying out usability testing does not always mean that you’ll get the product done right. Sometimes, mistakes are made along the way, and they affect the test results. And if the development team based their efforts on the skewed results, they’ll end up with a product that’s a bad match for the users.
I must admit that I’ve committed my fair share of mistakes during my years of product design. In this article, I’ll share those mistakes so that you’ll be aware of them when conducting usability testing.
Usability Testing Mistake 1- Not Setting Intentional Goals
Usability testing is the procedure that helps to crystalize real user’s needs and problems. Each of such tests should aim to discover one specific user need. At this point, it’s easy to read up a testing strategy online and implement it blindly on a project.
Without setting proper goals, you’ll run the tasks aimlessly and receive results that are not helpful at all. You couldn’t determine if the product matches what the user wants if you don’t have a goal to start with.
How To Avoid This Mistake
Remember that usability testing is a purposeful activity. It is carried out to validate your team’s assumptions regarding whether this or that need is relevant. These assumptions are professionally called “hypotheses” – these are your intuitive conjectures on whether the problem exists or not. The user test’s purpose is, therefore, to validate where these assumed problems are real.
Usability Testing Mistake 2 – Not Setting the Right Tasks
If you fail to set clear and proper tasks for the participants, they’ll encounter various problems during the test. They’ll be fiddling all over the user interface without accomplishing anything of value.
Sometimes, you’re setting incorrect tasks that don’t provide the data needed to validate prior assumptions. For example, the participants are asked to add items to the shopping cart, but the real issue that needs clarity is the discount voucher selection.
How to avoid this mistake
Your tasks for the participants should stem from the needs you have. So before running usability testing, take your time to figure out the blind spots of your product’s interface that need validating.
Also, think of how the tasks need to be designed to produce accurate measurements and results. The best way to organize your findings is using a test card as a guide for the participants. The participants carry out the task accordingly and fill out the test card in the process.
Usability Testing Mistake 3 – Recruiting the Wrong Participants
Imagine that you’re building a video-sharing app like TikTok, but you’re seeking out participants in their mid-40s who have zero interest in such apps. It doesn’t take a wizard to predict that the result will be useless. It’s a pure mismatch between participants and the intended users of the app.
Unfortunately, recruiting the wrong participants is a common mistake in usability tests. It happens because the team fails to establish the target audience or “proto persona” for the product’s potential users. This often leads to a helpless outcome, particularly if the test data is used for development.
How to avoid this mistake
Your team needs to brainstorm about who the product will appeal to. To this end, our team makes a proto persona – an assumed image of who our user might be, including their interests, age, hobbies, profession and other characteristics relevant for the app development. Here is a template of a proto persona document that we use.
To prevent getting the wrong participants for the test, you’ll need to go beyond demographics.
You’ll need to question if the persona is familiar with similar products. It helps in getting more engagement from those participants during the test, and thus, more accurate results.
Once you’ve determined the participant’s criteria, you can post your requirements on Craiglist or use an online tester panel like TryMyUI.com, or UsabilityHub.com
Usability Testing Mistake 4 – Guiding Users Through a Test Process
There are times to be helpful, and there are times to be not. You certainly don’t want to be helpful during a usability test. The purpose of such a test is to learn how the participant responds to the product. It’s the best opportunity to capture their emotions, thoughts, and feedback. If you give in to the participant’s plea for help, you risk compromising the test results.
How to avoid this mistake
Make it plain and clear for the participants during the briefing that test facilitators will not provide any sort of guidance on the tasks. Let them know that there is no ‘right way’ in accomplishing the tasks. The participants are to record down the process, including when they’re stuck, and their feedback is valuable.
Usability Testing Mistake 5 – During Usability Testing, Respondents Forget To Voice Their Thoughts
If the participants are a tad silent during the test, you’re going to have problems in the interview that follows. The participants will have trouble recalling their interaction with the product if they’re thinking aloud instead of speaking out when testing.
Therefore, it is important to ask the participants to voice their thoughts as they work through the task. You can use video recording and revisit the clips later if the participants need to refresh their memories.
How to avoid this mistake
Do let the participants know that it’s important to be vocal in their thoughts when interacting with the product. Encourage them by having them describe what they see on the screen.
Usability Testing Mistake 6 – using design terminology when talking to a respondent.
Developers can get carried away by using terminology when conversing with the participants. Terms like ‘UI’, UX’, and ‘Sprint’ are words specific to the IT community but not the typical participants. When you’re using such technical words, the participants will struggle to understand what is actually being conveyed.
How to avoid this mistake
You need to be mindful that you’re not talking to a group of developers but people who are not technically trained. Choose common words when speaking to the participants and ensure that they understand what you said.
Usability Testing Mistake 7 – Conducting Only One Usability Testing Phase
It takes a few stages before you get a functional product. If you have conducted only one usability testing during those stages, you’re putting the product at risk. You shouldn’t treat the results of a single test to be definitive and free from errors.
There can be a degree of error in test results and developers may veer off target during development. A single round of usability tests is never enough. You don’t want to end up with serious bug issues after the product is built.
How to avoid this mistake
Strategically increase the number of usability testing. At the very least, you’ll need to have two different tests at the prototyping and the UI design stage. Running two tests before the development starts reduce the risk of producing a problematic product.
Usability Testing Mistake 8 – You Test To “Confirm” Your Ideas
As humans, we’re subjected to confirmation bias, where we tend to lean towards certain ideas. We fail to be objective in usability testing and devise the entire test based on our preference.
When you turn usability testing into an idea validation process, you’re defeating its purpose. The test is meant to measure and observe how users interact with the product. It isn’t about complying with your expectations.
How to avoid this mistake
You’ll need to be objective and stay neutral when conducting usability testing. It’s easier said and done, but you can start by willing to accept that you can be wrong too. Keep an open mind and be ready to trust data to speak for itself. Make it a team effort when analyzing the results.
Summary
For all its usefulness, carrying out usability testing isn’t easy. Chances are, you have made some of these mistakes when conducting usability tests. I hope that the above tips are helpful to avoid making the same ones in the future.
Where To Learn More
The Interaction Design Foundation’s online course on Conducting Usability Testing is a great place to learn how to:
- Conduct an effective usability test
- Configure a usability test and set test goals
- Build a usability test plan using best practices
- Carry out a usability test smoothly and effectively by avoiding common mistakes
- Report on your usability test observations
If you’re new to UX Design, then consider taking a membership with the IxDF, where you can access over 30 courses that dive deeper into the aspects of design, such as Design Thinking, or User Research – Methods and Best Practices, Data-Driven Design, and more.
Make sure to explore their free library on UX Literature here.
Good luck on your learning journey!
