An important aspect of building world-class products is to take feedback from users while you’re building it. This is done to constantly improve upon the product as per what the users want, which ultimately helps in successfully crushing the business goals. Usability testing deals with making users interact with your product or website to evaluate its user experience. It helps in determining whether users can use your product with ease or not and also know about possible usability issues overlooked by designers, UX researchers, developers, and product managers.
Types of Usability Testing
Usability testing can broadly be classified into 3 domains:
Moderated vs Unmoderated
As the name suggests, moderated usability testing involves moderators who moderate the testing session, guide testers, answer their queries, take feedback, ask follow-up questions, and make the testing process as smooth as possible for participants. Moderated usability testing is expensive as it requires time, effort, and resources, but it provides an in-depth understanding of usability issues.
There are no moderators to guide the testers during unmoderated usability testing. The testers could be testing from their homes or a lab. This type of usability testing provides little to no feedback, especially during testing complex workflows as the participants mostly give up on the tasks. If you want to observe and measure behavior patterns, go for this type of testing.
Explorative vs Assessment vs Comparative
Explorative tests are done during the initial stages of product development and are open-ended. It requires participants to share their thoughts on the product concept and its usability. It also helps in understanding the gaps in the market and potential new features.
Assessment tests are used to evaluate user satisfaction and a product’s general usability – how well the participants can use the product.
Comparative tests are primarily aimed at comparing two websites or products. It involves participants to choose a solution over the other.
In-person vs Remote
In-person tests are done in the presence of a moderator and the session is more fruitful than remote testing sessions as more data points like body language are obtained. Do not go for this type of testing if you want to test with a large number of participants. Generally, in-person tests are costly, require space, and are time-consuming.
Remote testing is done on the phone or over the internet. It is cheaper and more tests can be conducted when compared to in-person testing. The only problem with the remote testing process is that fewer data points (or usable feedback) are obtained. Let’s dive deep into remote usability testing below.
What is Remote Usability Testing?
As mentioned above, with remote usability testing, you can conduct user research with participants in their natural environment. You must share your screen with the participants or employ software from vendors to conduct remote usability tests. When the travel budgets are small, timeframes are tight, or participants are hard to find – go for remote usability testing.
Rather than the session facilitator meeting the tester in person or vice versa, the tester performs the tasks from their home, office, or any other location, and the facilitator (or group of experts) watch the tester performing the tasks remotely.
Moderated & Unmoderated Remote Usability Testing
Moderated usability testing involves the presence of a facilitator who guides the participants during the testing process. The facilitator watches the participants perform their tasks in real-time via Skype or other tools that enable screen sharing like Zoom, GoToMeeting, and Lookback. After the session is over, the moderator can ask follow-up questions from the participants.
Unmoderated usability testing does not require a facilitator to be present during the testing process. The participants take the test in their natural environments and are free to perform the tasks at their own pace. The results of the testing process are in the form of recorded videos. No follow-up questions are asked from the participants.
Remote Usability Testing Tools
Here’s a list of tools used for conducting remote usability tests:
- Zoom
It is a leading video conferencing software that enables recording and live screen sharing. You can signup for free on Zoom. Its user interface provides an amazing user experience. The only disadvantage of using Zoom for remote usability testing is that the participants could be reluctant to download another piece of software. - GoToMeeting
It is another web conferencing tool popular among remote workers. It boasts end-to-end encryption but it is difficult to set up and the interface often confuses the users. Moreover, it’s more expensive than other tools especially if you want to record session videos. - Lookback
It is built for the purpose of user research and can be used on any device. It can also be used for conducting self-moderated tests. The tool works well if you have a stable internet connection. The recordings are stored in the cloud. The tool captures the user’s screen and webcam, and it also allows you to tag the timeline with notes. The only disadvantage of using Lookback is that the user needs to install a software/plugin. - Userzoom
From videos to mobile surveys, this tool provides different methods for remote usability testing. It even allows you to recruit your own set of participants for the testing process. The user interface may feel a bit outdated at times.
Benefits of Remote Usability Testing
- It is generally less expensive than in-person testing.
- You can conduct tests on a lot more number of participants when compared to in-person testing.
- It is less expensive when compared to traditional in-person testing as the latter requires a lab environment.
- You can accommodate diverse groups of participants which will give you a better idea of how each group performs during the test.
- A lab is not required for conducting remote usability tests.
- With remote usability testing, you can get insights on how your product performs in various geographies.
- Finding new participants online is less time-consuming and cheaper than finding participants for in-person testing. Remote usability testing is scalable.
- The insights you gain from testing remotely are closer to the real-world as the participants take tests in their natural environments thereby nullifying the Hawthrone effect.
- In moderated remote usability tests, you get more data points as you can see participants performing the tasks in real-time.
- Moderated remote usability tests provide additional flexibility as you can test and observe the behavior of participants in real-time. Additionally, you can clarify the question in case of a misunderstanding.
- A large number of participants are recruited for unmoderated remote usability testing. This helps in confidently generalizing the findings.
Challenges of Remote Usability Testing
- Without the presence of a moderator in an unmoderated remote usability test, the participants may simply leave or do not pay much attention to the tasks.
- The observer effect is still present during moderated remote tests even though the participants take the tests in their natural environment. In the presence of a moderator, the participants may not speak up even if they find it hard to use the product.
- Moderated usability tests are generally time-consuming. The moderator has to be present at the test location all the time in order to not miss out on interacting with the participants and the chance to ask follow up questions.
- The accuracy of the results of an unmoderated remote test largely depends upon the clarity of the scripts as there is no moderator to guide the participants through the testing process. The results are generally in the form of screen recordings.
- After the completion of an unmoderated remote usability test, no follow up questions can be asked from the participants.
- Some participants take moderated tests just to make quick money. They may not provide valuable information to you. Sometimes, they may say things just to please you.
Tips for Remote Usability Testing
- Recruit more users – The number of participants who do not show up for a remote testing session is far more than those who actually show up during in-person tests. Therefore, you must recruit more testers for remote usability testing.
- Mitigate the observer effect – Make the participants as comfortable as possible. Ask them to speak up when they are not able to use the product or any of its features. Tell them that you don’t mind them speaking badly about your product or any of its features.
- Clarity – Ask clear and concise quality non-overlapping questions during an unmoderated remote usability test. There must not be any ambiguous tasks.
- Filter out unwanted recruits – Filter out those recruits who are just in for the money as their inputs will not provide any considerable insight with respect to the user experience of the product.
Conclusion
Remote usability testing is cheaper than in-person testing. A large number of participants can be recruited. However, the remote testing process comes with its own set of challenges. The insights gained from remote testing are closer to the real world. Conduct moderated usability tests if you want to see the participants taking the test real-time and if you want to ask follow-up questions.
Want to learn more?
If you’d like to…
- get an industry-recognized Course Certificate in Usability Testing
- advance your career
- learn all the details of Usability Testing
- get easy-to-use templates
- learn how to properly quantify the usability of a system/service/product/app/etc
- learn how to communicate the result to your management
… then consider taking the online course Conducting Usability Testing.
If, on the other hand, you want to brush up on the basics of UX and Usability, then consider to take the online course on User Experience. Good luck on your learning journey!
(Lead image: Hotjar)
