Imagine a usability test in which John, the test participant, attempts to buy a bicycle. On the homepage John quickly finds the “bicycles” link, but on the next page he hesitates. “I wasn’t sure where to click,” he says afterwards, “there were a lot of options.” Later, while reviewing their notes, the test facilitators take everything John said into consideration. They note where his mouse hovered. They check how far down the page he scrolled. One of them comments, “I wish we could see what he saw.”
Eye tracking is sometimes perceived as a silver bullet – the ultimate technology with which to diagnose user interface issues. It’s no surprise, then, that eye tracking has been growing in popularity for over 20 years.
But behind every impressive demonstration is hours of effort and interpretation. Some of the industry’s foremost experts remain conflicted as to its value, and it’s still only undertaken in a minority of web design projects. Despite plummeting costs and increasingly reliable technology, the costs of time, training and equipment are still substantial. So the real question seems to be: what can you get from eye tracking, and when is it a worthwhile expense?
Eye tracking … in a nutshell
Eye tracking is, simply, the observation and recording of eye behaviour such as pupil dilation and movement. It has applications in many areas, including psychological research and packaging design, but with regards to screen-based media, it’s primarily used by researchers to identify where users are looking.
Of special interest are points of “fixation”. These are areas in which a user’s gaze stops moving, lingering long enough for them to process what they’ve seen. The movement of a user’s eyes between fixation points is known as a “saccade”. Although the speed of the movement means the user is not processing what they are seeing, visualising saccades shows the path the eyes are taking between fixations.
A common method of calculating the focus of the user’s gaze compares the position of a near-infrared light (reflected in the eye) to the position of the pupil. This information, combined with the position of the participant’s head, can be extrapolated to determine the point the user’s eyes are focused on, and thereby the corresponding screen coordinates.
Eye-tracking devices construct a dataset by recording coordinates many times a second. That dataset can then be visualised and interpreted to expose behaviour that is otherwise invisible, including:
- An ordered list of fixations (and an unordered list of overlooked elements): First and foremost, eye tracking highlights both what the user sees and what they don’t see.
- The time taken to arrive at any given fixation: This can be related to how easy or difficult an element was to find.
- The length of any given fixation: This can be related to how engaging or comprehensible an element might have been.
- The number of fixations per element: This can be related to how distracting, useful or contradictory an element might have been.
Seeing is believing
Eye-tracking devices don’t depend on participant reports or memory. When questioned about their actions, test participants may not remember their behaviour. They may be unaware of what they did (as it was subconscious or forgotten), or they may simply be unable to verbalise their reasoning. Through examination of data, visualisations and replays, the causes for behaviour can be found without relying on the fallible human memory. Crucially, eye tracking analysis can lead to discoveries that would be considerably more difficult to uncover with other testing methods.
In a recent study of a browser-based game prototype, for example, our eye tracking team at Cyber-Duck tasked participants with finding the record of their performance in earlier chapters of the game. To do so, users would have to click one of the “Review” buttons, under either “New York” or “Buenos Aires”, to proceed.
We knew from our observations that only half of our participants completed the task successfully. Conventional usability tests would tell us as much, and that alone would be enough information to recommend solutions to the problem. However, eye tracking provided us with more detail on the problem.
Although participants looked at the “Review” button often, they seldom clicked it. From this we learned that users were, in general, aware of the button. The fact that users looked at the correct button but did not click suggests that they did not understand its purpose.
The gaze plot pictured above shows the behaviour of a single participant. From the many fixation points and saccades we can tell the participant looked back and forth between elements multiple times. This suggests they weighed each option, unsure which was the correct way to continue the task.
With other testing methods, we would have discovered the larger usability issue: half of our participants were unable progress past this page. With our eye tracking study, however, we achieved greater specificity, noting that half of our participants were unable to progress past this page despite the fact that most of them looked directly at the button required to move forward. In other words, attention was not the issue. Accordingly, we refocused our efforts towards more clearly communicating the button’s purpose.
With the enhanced insight into user behaviour provided by eye tracking, problems (and subsequently their solutions) can be better targeted, saving time and effort.
Is eye tracking worth it?
An eye tracker is a tool and, like any other tool, it is most effective in the hands of an expert. Real insights can only come from a facilitator with the knowledge and experience to analyse the data eye tracking provides.
Eye tracking is not a replacement for other, qualitative research techniques. The data from an eye tracking study may indicate that users spend more time fixating on a certain element, but it can’t tell us why; this qualitative information can only be discovered through interviewing, observation and experience.
Though by no means essential, eye tracking can improve our understanding of the behaviour a user exhibits during a usability test and, therefore, allow us to create more targeted solutions.
Before jumping in to eye tracking, it’s worth considering whether you are utilising less expensive testing methods to the best of their potential. If the answer to this question is yes, eye tracking is a natural next step. If the answer is no, the introduction of low-cost methods such as observation, user interviews and A/B testing is enough to provide a great leap forward.
As Nielson and Pernice say in their book Eyetracking Web Usability, “you can double the profitability of your web site… with no other equipment than a notepad”.
Getting started
Assuming you are utilising less expensive techniques to their maximum and are ready to get the depth of understanding that eye tracking offers, there are still variety of ways that you can go about it; each with significant implications in terms of costs and the output you’ll obtain.
Scope
Eye tracking studies take time. For qualitative eye tracking tests where recordings are manually reviewed, five users will suffice, but it’s necessary to recruit at least 39 participants for meaningful heatmaps and other visualisations that aggregate the actions of many users. The costs of recruiting and incentivising participants should be part of any user testing budget or external quote, but it’s important to weigh up the significant time investment necessary to test with this many people. At Cyber-Duck, we’ve also found it beneficial to have two facilitators present in each session, to ensure nothing is missed and allow each to take a break. With individual tests lasting 20 minutes to an hour, this can quickly add up.
In-house or outsourced
The question then remains whether you set up an in-house eye tracking team or outsource your eye tracking to expert consultants. Due to the significant up-front investment required, it’s only recommended to setup an in-house eye tracking team if:
- You have staff with the right background for eye tracking training
- You have the resources to purchase equipment and train staff to use it
- You Will be running eye tracking studies regularly
- You are sure that the increased depth of findings will be worth the time and expense
Falling costs can make in-house eye tracking an attractive prospect, but there is much to consider before making the investment. Eye tracking is not (yet) as simple as turning on the eye tracker and pointing it at a participant. Besides calibrating the equipment, a comprehensive understanding of eye tracking methodology is fundamental for designing studies and analysing results. Without this, significant behaviour could be overlooked or misdiagnosed. In the worst cases, results could be entirely skewed.
Though the price has fallen considerably in recent years and continues to do so, expect to pay £10,000 or more for professional grade hardware. Software is often included but will likely require a powerful computer, which is an extra expense to consider. Weighing the merits of hardware and software options is a subject for another article, but products from the manufacturers EyeTech, Mirametrix, SMI and Tobii are popular choices.
If the expense of training, hardware and time is insurmountable, or eye tracking is rarely called for, you can still utilise the best technology by building a relationship with a consulting firm, and cost effectively reaping the benefits of eye tracking.
Search engines will no doubt return the best local leads, but suitable agencies can be found via LinkedIn eye tracking groups or the member databases of organisations like the British Interactive Media Association (BIMA), Interaction Design Association (IXDA) or User Experience Professionals Association (UXPA).
When assessing a potential eye tracking partner, case studies and client testimonials can be useful for gauging competence. Look for evidence that their studies and recommendations have resulted in improvements to key usability metrics like success rate, task completion time, error rate and user satisfaction. Are they able to communicate their findings in a way that is comprehensible to those outside of their industry? Jargon may sound impressive, but ultimately they must be able to justify their recommendations in plain language. For more advice, consult the UK UXPA guide Key questions to ask your usability testing supplier.
Final thoughts
Like any evolving technology, eye tracking has the potential to come into everyday use as progress eliminates barriers to entry. For now however, the considerable costs mean its return on investment will be greatest for those who have already reached the limitations of cheaper, low-tech user research techniques.
Want to learn more?
If you’d like to improve your skills in User Research, then consider taking the online course User Research – Methods and Best Practices. The course certificate is recognized by industry and it will help you advance your career. Alternatively, there is an entire course on Usability Testing which includes templates you can use in your own projects. Lastly, if you want to brush up on the basics of UX and Usability, you might take the online course on User Experience. Good luck on your learning journey!
(Lead image: Depositphotos)
