One of the main pillars of the user-oriented design involves testing your own assumptions and design solutions. Usability testing is very important in the design process. It allows you to verify ideas, helps to reduce the costs already at the initial stage of design and allows you to explore the user expectations. These have significant meaning when it comes to the user satisfaction.
Recently, remote usability testing is becoming one of the main ways of conducting a usability testing. It offers significantly more opportunities to reach out to the users and collect more data than in the traditional laboratory testing. Geographical spread, different target groups, the natural environment where the test is conducted – these are just some of the incredible advantages of the remote research. Despite that – in order to utilize the full capacity of the remote usability testing – you must carefully choose the method of recruitment and use it wisely. And yes – it can be fun and exciting! So let us have a look at six different ways to recruit participants for remote usability testing.
1. Recruiting participants from a panel
This is one of the most commonly used recruitment channels when it comes to remote usability testing. Implementation is simple – you need to purchase a given amount of responses by respondents from a service provider. Recruitment of participants from the panel is a universal way, which is perfect for testing usability of the mock-ups and prototypes, performing benchmark studies, testing information architecture, and many, many others. The format, presentation and length of the study do not impact the recruitment process in this case.
The advantage of this approach is the relatively low cost and rapid implementation of the study. The panels offer very different prices per sample. Some providers calculate it based on the complexity of the research, its duration, defined target group and place of origin of the respondents. Other companies charge a fixed amount for each participant without looking into the length of the study. If we decide to pay the service provider, we get virtually guaranteed that the required number of samples will be collected within 2-3 days. Another advantage is the ability to quickly reach geographically dispersed groups, which can be defined by the variables in the panel. Some of the filters that you may want to use are: annual income, number of children, tenancy status etc. This is a very attractive and fast approach.
Unfortunately panels also have disadvantages. One of them is the consistency of the respondents. In some cases these are the people who participate in tests and polls and use it as a source of income. This situation may lead to poor quality of the collected data and may mislead us. To protect yourself against it, always ask where do the respondents come from, whether their numbers are increasing or decreasing. It is also good to know how many surveys does a single participant answer each month (or at least how many does the most active one respond to). This will allow you to build the trust in the data gathered for you.
2. Recruiting Participants from your Site users – Automatic Invite
My personal favourite – down the line it is the actual end users’ perception that counts. If you already decided to use automated invites, make sure that the test is quick and easy. Think carefully about each of the questions, ensure that there are no duplicates and that the survey is comprehensive. You are now walking on the thin line between user’s time and the quality of the data that you gather.
User’s time is precious. I can assure you that none of them are keen to waste it answering questions that they see as unnecessary or repeated ones. In my personal opinion this type of the remote test should not take longer than 5 minutes to complete. User should be able to answer to up to 15 questions in this time slot. These should be the key “burning” problems.
This study requires both very well thought-through content and the presentation. Unsurprisingly – some elements can easily put users off and give them a reason to abandon the test. One of the best examples could be the question with the Likert scale. It normally consists of 30 sentences and the user is asked to express his opinion about them. UX researchers love it – it does provide them with a valuable data. Unfortunately our study shows that the users are usually discouraged the moment they see this type of a question and typically withdraw from the test. Avoid it – instead create simple, short and easy to understand questions.
One of the main advantages of this recruitment method is that you get the feedback from users who are familiar with the product. In many cases they are also involved in the development of your product and their minds quite often contain wonderful ideas. Therefore do carry out a satisfaction and engagement survey alongside with evaluation of the solutions implemented. Usability issues should be left to the panel participants. Use your customers’ feedback to gather information for the development of your product and their customer satisfaction.
Another advantage here is the low cost of recruitment compared to other methods, although it is not equal to zero. This study requires a lot more work, precision and careful execution. These take more time and requires engaging entire division of the company.
The disadvantage of this method is that collecting the data can sometimes take weeks. Especially if your website generates little traffic or you have a handful of customers. Another big disadvantage is that this test must be taken into account among other surveys and tests conducted by your business. It cannot be repeated too often either. You do not want your customers to feel cornered by your constant questions. Therefore try to prepare for such test in advance. Identify the most pressing issues and be aware that you can only ask these questions once for the next few months. So do your best to best to make the most of the 5 minutes that the users are giving you!
3. Recruiting Participants from your Site users – Competition
A small competition using social media is an interesting alternative. You will have to offer a reward for completing the test/survey. Make sure that the prize is interesting enough to convince your users to participate. Remember that the more valuable and interesting (from the user’s point of view) prize is – the more detailed can the question/interview be. You still have to present your survey in an attractive way, with well defined questions. You are still dealing with your users, so treat them seriously.
If you happen to be running a company that is offering Software as a Service (SaaS) you can offer free access to your service, a discount or another prize. In this case engage your marketing department. They will help you to run the competition, prepare the catchy slogans and advise where necessary.
Good ways to advertise this type of test is through your website, blog or the social media.
Your only limit is your imagination …and your company’s management!
The competition approach works best when used to test new functionality or to gather information about the user’s experience in the current version of the system (or modules – such as the knowledge base, transaction processing etc.).
This recruitment approach is not free either. If you decide to use it, you are looking at the costs of the prizes, the time that you will spend preparing the terms and conditions and managing the competition.
Last, but not least, coordination, the cooperation of the different departments involved, and planning are the activities that are indirect costs that you should remember about.
There are many advantages of this recruitment approach – among them, the fact that you are testing the existing users. Also, the actual cost of the entire process is comparable with hiring a third party to conduct the tests for you.
Just like everywhere else – there are disadvantages. One of them is the uncertainty. One would not be able to forecast how many responses will they will get. Also, it is difficult to predict how much time will it take them before they can see the first results. It may be a week, two – or just a day. A big disadvantage here is the amount of work that you need to do in order to recruit the users, engage other people and actually use the gathered data to plan the next steps of the R&D of your company. The dates should be planned in advance too, alongside with the research strategy.
Nevertheless it is worth it and I highly recommend it. When compared to recruiting site visitors through automated invitation, this recruitment method allows you to verify your existing solutions and test the new ones in much more detailed way. This is because the users have more motivation to participate.
4. Recruiting the participants through a Research Agency
This method is also a way of recruiting participants and is mostly suited for complex research that needs to be conducted on a very specific target group (which cannot be reached through the panel users). An example could be professionals such as lawyers, medical doctors, surgeons or music composers who use tool X. This type of recruitment is obviously more expensive. The process typically start from $100 per user. Again – the price depends on the number of survey responses that we are looking for, the type of the target group and time that is required to complete the test. In addition, the price can be impacted by the type of vendor that one decides to use, their brand, the area where they operate or even the length of cooperation with them. To sum up – the price per survey is directly linked to the test and the vendor who provides the participants.
These companies make sure that the data you get from their participant is accurate. One can always come back to the participant and ask additional questions. We can also define the error level in the data. Incorrect data will be replaced with the new responses. Working with an agency offers the comfort of detailed testing and tuning the data received. As I have said, this comfort does cost quite a lot. But – you do get what you pay for.
5. Recruiting the participants yourself
If you happen to work for a research agency and conduct many tests and surveys, it makes sense to have your own respondents’ database. It allows you to test test-drive your ideas before asking a wider audience to participate in your test. Alternatively you may choose to conduct a sophisticated test that – if done through an agency or panel – might be very expensive. Being able to test-drive your research strategies allows you to verify the initial response and adjust it if required.
This approach is also cost-effective. All it takes is few hours of your time to recruit people and pay them their compensation. It is also important to mention the time that you need to spend defining the strategy and managing these tests. Good organisation and project management skills are a must for this type of participant recruitment.
Unfortunately – there are disadvantages too. Having your own test group comes at a cost – that of running your own database and recruiting the new participants (which may turn out to be expensive). One more thing to think about are the results of the test – you need to ensure that they are reliable. Should any issues arise – you are the one who will have to deal with them. If a responder is late or fills the survey incorrectly – you will have to follow up with them and clarify the issue. In some cases you will have to advise them that they might not receive compensation for the survey due to the poor data quality.
Despite the challenges having your own testing group is well worth considering. It will add a lot of value to your development process and will save you money in the long run. Needless to say, having testers who are committed and familiar with your product will increase the quality of the data you can obtain.
6. Using your own resources is also a good idea
This technique is especially applicable to the big organisations. You can start testing your solutions using your internal resources. The quality of feedback gathered from your colleagues and employees is not different from the one received from paid testers (it is actually better on some occasions).
The incredible advantage here is that you can choose to repeat the tests more often than in the other cases described in this article. In many cases the testing scenarios are complicated and require a lot of time in order to explain them to the users. If you use your internal resources, who already familiar with the project (or product) they will be able to understand the idea much quicker. The costs are lower too.
As if that was not enough – you can always follow up with the testers and ask them additional questions. While working with a survey there is always a chance that the answers received from some of the participants are not clear. In this case the ability to come back to to clarify the details is priceless.
Like everywhere else – you have to remember that your employees are people too. They do get bored. And they do have their jobs to do so you cannot bother them all the time with the UX tasks. If you do – it may affect their productivity and the data quality along the way.
Conclusion
Make sure that you always choose the right method of recruitment for your research. When it comes to the remote testing the recruitment is the key to success. Check if you can follow-up with the respondents if you need to.
Remote usability testing allows you to achieve reliable results and you can use it to design your development strategy around it.
Want to learn more?
If you’d like to…
- learn all the details of Usability Testing
- get easy-to-use templates
- learn how to properly quantify the usability of a system/service/product/app/etc
- learn how to communicate the result to your management
… then consider to take the online course Conducting Usability Testing.
If, on the other hand, you want to brush up on the basics of UX and Usability, then consider to take the online course on User Experience. Good luck on your learning journey!
(Lead image: Depositphotos)
