Eye-Tracking that Reads Emotions! Feelings drive much of our behaviour and so emotional engagement with digital content can be a strong indicator of its effectiveness at getting our attention. This is especially critical for mobile gaming apps as unless users are engaged they may lose interest and delete the app from their phone. Imagine then the power of emotion-enabled gaming apps that recognise emotional responses in real-time and where these same emotions can modify games dynamically to create a more engaging and exciting user experience.

Source: Affectiva.com

This is now becoming a reality with eye-tracking solutions that use webcams and device cameras to monitor emotional engagement. These solutions use vision algorithms to identify key landmarks on the face, such as the tip of the nose or the corners of the eyes and the mouth. Machine learning algorithms then analyse pixels in those regions to classify facial expressions based upon pixel colour, texture and gradient.

Combinations

Combinations of facial expressions are then mapped to emotions and data is processed in real-time through SDKs. This data can usually be accessed via APIs or visualised in a dashboard. Indeed, eye-tracking solution Affectiva’s SDK integrates into user interfaces to allow developers to emotion-enable their apps and digital experiences so they adapt to people’s emotions in real-time.

Source: Affectiva.com

This creates new opportunities to optimize digital content in a more sensitive and engaging way. Apart from game design, the technology is also especially valuable for usability and market testing, video and ad evaluation, and website design optimization in general.

Interestingly though research by Nielsen indicates that facial coding used in isolation is a poor predictor of sales. EEG research that tracks brain waves is a more powerful indicator. This is why it’s important to use emotional engagement measures along side eye gaze and other research methods. It gives you a better understanding of the impact of visual content.

An Advantage

One advantage of eye-tracking though is that the eye doesn’t lie. We can’t control what we initially look at when we see an image. Indeed, research by eye-tracking company Sticky.ad has discovered a gap between what people say they were looking at when they first glance at an image and where their gaze is actually focused. Self reporting is an unreliable method of identifying what people look at partly because our conscious brain filters out a lot of the information that our visual cortex processes.

Psychologists have also discovered that we don’t have full access to what motivates a lot of our behaviour. Our attention is largely determined by our subconscious brain as it scans for solutions to implicit or psychological goals.

Source: Sticky.ad

At the same time usability testing and remote Voice of Customer solutions can add to our understanding of how people interact with a user interface. It provides insights into the issues they encounter. Each method of research provides a different perspective on how people respond to visual content.

Is There A Correlation Between Eye Gaze and Mouse Movements?

Some user experience solutions claim there is a high correlation between mouse movement and eye gaze, but the evidence does not support this. The highest published correlation is 69%. But research by Google (see page 29) indicates it is more in the region of 42% and according to an anecdotal study it is only 32%. However, Simple usability claim it may be even lower, if not non-existent, when visitors begin to scroll. This is because people tend to scroll and scan content with their mouse either being static (because they use the scroll-wheel) or their mouse follows the right-hand scroll bar. You can almost guarantee that users won’t be looking at where their mouse is when they scroll down a page.

Source: sticky.ad/

The “F” Shape:

Source: Gazehawk.com

Many eye tracking studies have shown that users tend to scan web pages in an “F” shaped pattern. This is partly because in the West at least we read from left to right. As a result we mainly focus on the top left side of the screen and scan across horizontally. Our eyes then scan down the page, again mainly down the left to find what we are looking for. Of course this process can be disrupted by content that grabs our attention or design elements that separate content. Web pages for instance that appear to end at the bottom of the screen won’t encourage further exploration and so engagement often falls dramatically around the fold area.

Google Golden Triangle & Beyond:

In 2005 an eye tracking study by Google and marketing services companies Enquiro and Didit.com, discovered that users tend to view Google search results in more of a triangular shape. Thus a majority of a user’s eye gaze is focused in a triangle at the top of the search results page. This became known as the “golden triangle”.

Source: Mediative

However, Google conducted a further eye tracking study in 2015 which was published by Mediative. This indicated that more visitors are moving outside of the ‘Golden Triangle’ and that the rise of the mobile user is also changing browsing behaviour.

Source: Mediative

The latest Google study shows:

• Mobile users scan vertically more than horizontally.
• Users are viewing more search results within a single session. But are spending less time browsing each one (on average 1.17 seconds compared to 2 seconds in 2005).
• Results that are lower down the SERP (particularly positions 2 to 4) are receiving more clicks than they did a few years ago.

What this tells us it that we can’t rely on generic eye tracking studies to inform us where visitor gaze is concentrated. We need to undertake our own studies where we can. Otherwise we will making decisions based upon assumptions that may well not be valid.

Observer Effect:

The Hawthorne effect refers to the fact that people often change their behaviour when they are aware they are being observed. This is why TV viewing research panels ignore what new participants watch in the first few weeks after joining the study. Viewing habits have been shown to change immediately after someone becomes involved in such research. It is therefore important to allow users to acclimatize to the study and if possible use equipment that is not too evasive.

Another way of reducing potential bias is not to disclose which website or page you are interested in and let the user start the study on another website. You can then guide the user to the relevant website through a search result or link and allow them to browse the page before moving onto another.

Emotion Sensing Eye Tracking Solutions:

 1. Affectiva:

A cloud-based eye tracking and emotional engagement research solution that uses a standard webcam or device camera. Developed using the world’s largest emotion database of more than 4 million faces across 75 countries, Affectiva has established norms to benchmark ad effectiveness by geography and product category. Unilever for instance uses Affectiva to measure emotion analytics for over 600 products in 70 countries. The solution also integrates with voice of customer tools and works with any panel provider.

The Affdex SDK allows developers to create interactive and emotion-aware apps and digital experiences that adapt to the emotional response of a user. The SDK monitors and reports on emotional responses from facial expressions by analysing the user’s face via the device camera, a video or even a single image in real-time.

Pricing: Quotations are available on request.

2. EyeSee:

An online eye-tracking and facial coding platform that uses the respondent’s own web cam to deliver fast, cost-effective and global eye tracking insights. Just before and during the eye tracking and facial coding research they monitor environmental factors and behaviour (e.g. head pose and head movement. If necessary it will recalibrate to ensure the accuracy of the research.

The platform is able to display a range of stimuli from static images, video and live websites. It can also make static images scrollable and zoom in by clicking on certain areas. For live websites the solution can activate or deactivate links and create overlays.

The facial coding functionality uses an algorithm to process webcam images which automatically identifies 7 emotions: Happiness, Surprise, Confusion, Disgust, Fear, Sadness and Neutral. The output from the algorithm is double checked by humans to validate the accuracy of the outputs.

Pricing: No prices on the website. Please contact EyeSee for a quote.

3. Sticky:

This is the World’s first cloud-based self-serve biometric eye-tracking and facial coding solution. Sticky utilises the user’s webcam and as a result the solution brings the cost of eye tracking and emotional engagement research within the reach of most medium to large companies. Further, results can be obtained within hours rather than weeks that it might take for a traditional eye tracking study.

The webcam eye-tracking software uses multiple tracking algorithms to identify the face, key features, eyes, iris and movement in 3D. This generates highly accurate eye tracking data and can be used to optimise all forms of digital content including images, ads, websites, videos and emails. Sticky’s Eye Portal allows users to identify areas of interest (AOI) produces data on three key metrics.

The Key Metrics

• The Seen Metric measures the percentage of the audience that saw the AOI.
• Stickiness informs how long on average was spent on it.
• Salience shows the average time it took a viewer to first see the AOI.

The solution is used for a range of purposes by Stick’s clients including website design optimization, ad placement valuation, article layout readability and more. Sticky also integrates with online surveys and it includes a feature to include survey questions within the portal.

Pricing: The cost is dependent upon the plan you sign up to:

• 6 months platform access, 20 media credits – $7,200 (10% off, $800 savings!) normally USD $8k
• 12 months platform access, 50 media credits – $14,875 (15% off, $2,625 savings!) normally USD $17.5k
• 12 months platform access, 100 media credits – $24,000 (20% off, $6,000 savings!) normally USD $30k

Each credit represents an image or video used in an experiment. Facial coding requires an additional credit per 30 seconds of video. Using the Sticky panel is extra, starting from USD $5 per participant. Using your own panel is currently free.

Eye tracking software to use in your own studies:

1. Eyegaze:

The Eyegaze System uses the pupil-centre/corneal-reflection method to track where the user is looking on the screen. An infrared-sensitive video camera, mounted beneath the monitor takes 60 pictures per second of the user’s eye.

A LED light mounted in the centre of the camera’s lens illuminates the eye. The LED reflects a small amount of light off the surface of the use’s cornea and pupil which reflects off the retina. This causes the pupil to appear white and enhances the camera’s image of the pupil. It makes it easier for the image processing functions to locate the centre of the pupil. The computer calculates where the user is looking on the screen based upon the relative position of the pupil centre and corneal reflection within the video image of the eye. This provides an average accuracy of within a quarter inch or better.

Before operating the application the system must go through a calibration process to learn several physiological properties of the user’s eye in order to accurately determine where they are gazing on the screen. This calibration process takes about 15 seconds. It does not need to be repeated if the user leaves the testing environment and returns later.

2. Gazepoint:

This desktop eye-tracking solution uses a camera mounted beneath the screen to calculate where the user is looking. It is ideal for UX and developers to conduct user testing to understand where on a page attention is focussed and how people navigate content. The tool offers eye gaze recording, Thinkaloud voice and webcam recording, heat maps, dynamic areas of interest, mouse clicks, pupil diameter data and where users stop scrolling.

3. Glance by mirametrix:

A non-evasive, portable eye-tracking system that sits just below the computer screen. It calibrates almost instantly and runs on Windows OS. It looks like an Xbox Kinect and can be used wherever you need to conduct user testing.

4. iMotions: 

Provide a comprehensive package of biometric and eye-tracking research hardware (including remote and mobile kit), software solutions and training. They use remote eye-tracking hardware and sell eye tracking glasses, biometric sensors and PC specs.

Algorithm based tools that predict user focus and attention:

1. Eyequant: 

Eyequant uses algorithms and machine learning to predict how visitors will see and perceive website pages. The software generates instant heatmaps covering attention, hot spots, engaged visitor analysis, perception maps, regions of interest, visual clarity and new visitor analysis. You can either enter a URL or upload an image for a new design.

Analysis is based upon eye tracking research, user studies and online experiments to understand how people perceive web designs. Statistical analysis is employed to identify correlations between specific design characteristics and user behaviours.

Design characteristics that are identified as influencing user behaviour are then used as variables in predictive modelling. Machine learning is employed to determine the optimal weight and combination of these characteristics. Predictions are regularly benchmarked against eye tracking and other user studies to measure the accuracy of the modelling.

2. Feng-GUI:

AttentionWizard from Sitetuners has partnered with Feng GUI to provide an evaluation of the visual effectiveness of landing pages by predicting what a person would most likely to look at.