How do you get quick design feedback so that you can minimise the risk of reducing conversion on your site or app? There is constant pressure to keep websites looking fresh and to add new functionality or content to improve the customer experience. However, from my experience of evaluating website performance one thing is guaranteed. Customers will always surprise you with how they do or don’t interact with a new site or app.
That killer functionality will rarely instantly take-off, if at all. Visitors will not behave as expected on your new website and they will often complain about the changes you have made. Key metrics will drop, and though they may largely recover, some measures may never be the same as on the old page or website. In some cases this may be welcome, but often conversion rates can suffer. So what should you do to prepare yourself for the launch of the new customer experience?
If you can you should A/B test your new design against the existing page or website. This will confirm how your key metrics are likely to change as a result of the new design. It won’t tell you why visitors are behaving differently. To answer these types of questions you need more qualitative design feedback rather than numbers. Below are ten tools you can use to get design feedback from customers or experts to help identify where users may be having trouble with your new customer experience.
Get design feedback from real people on your landing page, wire frames and mock-ups to understand people’s first impressions. Participants are given just 5 seconds to view your design. This helps you evaluate how intuitive your page is by understanding what a person can recall about your design based upon those first few seconds.
Price: $99 per month for access to all design feedback services.2. Loop11:
Online usability testing and design feedback. Get your first project free (up to 5 tasks and 2 questions). Covers over 40 languages, provides heatmaps and clickstream analysis and real-time reporting. You can also test on mobile devices.
Pricing: A Free usability test is available for new customers. Pay as you go costs $350 per project. All plans include 1,000 participants per project, unlimited tasks and questions, testing on mobile, real-time results and 24/7 email support.
The Micro plan costs $158 a month. It is designed for organisations with between 1 and 10 employees, plus for non-profits and public sector clients. The SMB plan costs $410 per month and is for 11 to 100 employees. The Enterprise plan is priced at $825 per month.
A forum of web designers and developers set up specifically to give design feedback from over 350,000 registered users. Sitepoint is a media company which serves the web design and development sector by publishing articles and e-books. Free membership provides limited access to the community, but for $99 per year you can get Premium membership which gives you access to over 5,000 videos, 83 eBooks, and live Q&A and chat with experts.
Provides design feedback from users through a customised feedback button for websites, apps and emails. This allows users to select the part of your website that they want to give feedback on and there are multiple targeting options.
Provides videos of users talking about what they think as they use your website. UserBob recruits people to visit your website. Set a scenario for the user and specify a task for them to attempt to complete. During visits respondents record their screen and voice as they think out loud about the experience. You then receive a copy of the video to learn about what users say about your site.
You decide how many users you need, what demographics match your visitors, and how long each one should spend on your website. The test is instantly made available for users to participate and you will normally have a video to review your design feedback within a few hours.
Pricing: Start at just $10 for First Impressions where 10 users
will spend one minute each on your website. Users will discuss their first impressions of your website, who they think it is for and what you can do on the site. Task Completion costs $20 for 5 users who spend 4 minutes attempting to complete your task. The price of the Custom test is variable. This involves between 1 to 10 users each spending up to 8 minutes with a specific scenario and user task to complete. You may also specify user demographics for Custom tests.
Get videos of users as they browse your website, app or prototype. Respondents describe their impressions as they complete agreed tasks and these are recorded together with their screens and mouse movements into online videos. UX experts then analyse and summarise the design feedback into high, medium or low UX issues.
A managed service offering is available which covers the WhatUsersDo research platform, instant access to an online panel from over nine countries, lab tests and UX experts to manage research and deliver insight reports.
Pricing: Pay as you go starts form just £30 + VAT per user and
includes tag videos, ability to download videos, download clips and PDF reports (including video). Prepaid Test Pack starts from £300 + VAT and provide for more cost effective user testing than the pay as you go plan.
Prepaid plans: All plans include design and scoping support from
UX specialists, expert analysis of results, and account management and email and phone support.
The Starter plan costs £10,000 per year for 50 video test
credits (1 credit = 1 completed video). The Repeat plan costs £20,000 for 100 video credits and the Regular plan is £30,000 per year for 150 video credits. An Enterprise plan is also available with 200 video credits – price available on request.
A free review and feedback tool that allows stakeholders(clients, managers, teammates, etc.) to add feedback with annotations, which can be converted into trackable tasks. These can be prioritised and assigned to team members so that tracking issues across the project is easier and faster. Teams can use zipBoard to gather feedback on images, HTML prototypes, PDFs and live websites.
To create a pain free user experience and optimise conversion it is essential to carry out usability testing research. Sure, you can ask a few people around the office to check out your new site, but it is more important to get feedback from ‘real users’ who do not work in e-commerce and are not connected to your business.
Psychology tells us that as we concentrate on a task or project we are prone to see what we expect to see because our visual cortex unconsciously takes the decision to filter out things that it regards as less important to achieving a task. This is why we often miss the most obvious mistakes if we proof read our own work. Usability testing removes this filter as it does not rely on our own opinions.
IKEA Effect
We also get too close to our pet projects and as a result we overvalue the things we create, (see the IKEA effect). As a result we are not the best people to evaluate websites that we helped to create. Anyone connected to your business may also suffer from some of the same cognitive biases or may just not want to hurt your feelings. Usability testing avoids these biases by getting the views of ‘real users’, that is people not connected with your organisation.
Source: Freeimages.com
Usability testing is not about proving or disproving something works or not. It is about informing decisions and giving you insights into how users interact with your website. If you need a definitive answer then you really should be conducting an A/B test. As I pointed out in another post on whether usability research is reflecting real behaviour all research is subject to bias and limitations.
People for instance change their behaviour when they are aware they are being observed. This can be a particular problem for usability testing which is conducted offline. However, it is possible to build procedures into usability testing to minimise this problem. For example you can get the user to complete a few fake tasks to disguise the real test experience.
When Should You test?
Source: Freeimages.com
The earlier you do some usability testing the better as this will allow you to respond to user feedback at each step in the development and design process. Wire frames, prototypes or even drawings can be tested to give you useful feedback before you move onto finished designs.
Don’t use focus groups as usability research needs to deal with one user at a time. Otherwise people can get distracted by what other people are doing and you also need to give them your full attention. It is important that you observe and listen to users and avoid asking questions as people will over-think their behaviour if asked to explain it.
How Should You Test?
Source: Freeimages.com
Steve Krug has written an awesome book, Don’t Make Me Think: A Common Sense Approach to Web Usability (Voices That Matter), which I highly recommend you read. He advocates doing your own usability research if you can. Undoubtedly this is a great idea if you have the time and the equipment. If you have no or very little budget you might as well do this as even one user test is better than none.
However, there are also some affordable online services available, some of which are free. The benefit here is that they can manage all the admin and recruitment for you, plus conduct the usability testing and if required do the analysis. All you have to do is agree the brief. I would still recommend you get videos of your usability tests as you will often learn more from watching people browse your site than from reading a report. A video brings it to life in a way a report cannot.
One other advantage is that such suppliers offer a range of solutions including remote automated usability testing, card sorting for developing your navigation, tree testing to evaluate how easy it is find content on a site and eye-tracking to identify where on page attention is drawn to. Such suppliers can also use their expertise to advise you on how to best design a usability test study.
Who Should You Recruit?
Source: Freeimages.com
Some usability test solutions, such as Hotjar, offer you the ability to serve pop-ups on your site to recruit your own visitors to complete specific tasks. You can then share screens using Skype or other web meeting tools to observe how successful they are at achieving the set task. Alternatively many suppliers offer the option to recruit testers who match your user demographics.
Don’t get too obsessed though with matching your target audience as usability testing is about understanding how people in general interact with your site. Setting very strict recruitment criteria will just increases the cost and time needed to conduct the testing without adding much value to the outcome.
Online usability testing with your first project free (up to 5 tasks and 2 questions). Covers over 40 languages, provides heatmaps and clickstream analysis, real-time reporting, and you can test on mobile devices.
Pricing: A Free usability test is available for new customers. Pay as you go costs $350 per project. All plans include 1,000 participants per project, unlimited tasks and questions, testing on mobile, real-time results and 24/7 email support.
The Micro plan costs $158 a month and is designed for organisations with between 1 and 10 employees, plus for non-profits and public sector clients. The SMB plan costs $410 per month and is for 11 to 100 employees. The Enterprise plan is priced at $825 per month.
Remote usability testing which provides videos of visitors undertaking set tasks on your website. You also get written answers to questions you set. Get your first test for Free – normally costs $35.
Pricing: The Personal plan charges $35 per test credit. A desktop test requires 1 credit, whilst a mobile test costs 2 credits. Includes up to 20 minutes of video and audio feedback, written responses to custom survey questions and the ability to analyse your results with tagged, time stamped annotations.
The Team plan costs $299 per month. This gives you 10 credits per month, testing with your own users for one month, multi-user login, collaborative video annotation, crowd sourced key insights with the UXCrowd, UX diagnostics and the ability to download your video results and test data.
The Enterprise plan is not priced on the website. However, this
includes 100 test credits per month, unlimited testing with your own users, extended 30-minute length for test results and one-click report generation integrated with video playback.
UsabilityHub have a great selection of simple but effective usability testing solutions. You can obtain first impressions of your mock-ups and designs, see where visitors want to click or discover how easy visitors find it to navigate your website.
Simply upload an image, and select the type of test you’d like to run.
You can choose from:
Five Second Test to understand people’s first impressions of
your design.
Click Test to find out where they click and how they interact
with your interface
Navigation flow test to identify how visitors navigate around your
website or applications.
Question Test – allows you to conduct fast surveys by uploading an image and asking users questions about the design.
You can then decide how many people you want to be in the test or even recruit your own testers. UsabilityHub then create a report showing a detailed breakdown of the interactions each tester had with your design.
Pricing: Responses from testers you recruit are free. Testers recruited by UsabilityHub cost 1 credit each and responses from testers of specific demographics cost 3 credits each.
The Free Community plan allows you to create unlimited tests, with responses from your own users being free and buy responses from UsabilityHub from $1 each.
The UsabilityHub Pro plan costs $99 a month and allows you to buy responses at 50% off all credit purchases, starting at just 50 cents per response. Create unlimited tests, customize the test experience with messaging and redirection after the test, use a single link for multiple tests in a row and target particular demographics.
Established over 25 years ago Usability Sciences offers a full managed service for usability testing, offering a comprehensive range of solutions including card sorting, rapid iterative testing, mobile & tablet user testing and eye-tracking research.
Watch videos of real users talking about what they think as they use your website. UserBob recruits people to visit your website. Set a scenario for the user and specify a task for them to attempt to complete. The user then goes to your website and tries to complete your task. During their visit they record their screen and voice as they think out loud about the experience. You then receive a copy of the video to learn about what users say about your site.
You decide how many users you need, what demographics match your visitors, and how long each one should spend on your website. The test is instantly made available for users to participate and you will normally have a video to review within a few hours.
Pricing: Start at just $10 for First Impressions where 10 users will spend one minute each on your website. Users will discuss their first impressions of your website, who they think it is for and what you can do on the site. Task Completion costs $20 for 5 users who spend 4 minutes attempting to complete your task. The price of the Custom test is variable. This involves between 1 to 10 users each spending up to 8 minutes with a specific scenario and user task to complete. You may also specify user demographics for Custom tests.
Get 5 to 15 minute videos of users recording their experience on your site and identify where the pain points are. Hear what users are thinking on their personal devices and in their natural environment.
Omni-channel usability testing solution. Will supply user testers from their panel or recruit to your specific demographic requirements. Alternatively you can recruit participants using a customisable invitation widget, by posting a link on blogs, websites, twitter, by using TaskRabbit, Mechanical Turk, Craiglist or by using a third party panel provider.
Userlytics allows you to test prototypes, videos, mobile apps, display ads, search and social behaviour, desktop and web applications, smart-phones and tablets and websites.
Pricing: Starts from $49 per user tester and depends upon the testing features you require, whether you need respondents recruiting, demographic needs, session length and reporting requirements.
Get videos in an hour of real people speaking their thoughts as they use your website, apps, prototypes and more.Do it yourself or access User Testing’s on-demand panel of over one million users to find an exact match of your target audience.
You can either select your users and write your own tasks or use the expect research team to complete such tasks as creating and managing tests, long term research road-mapping, moderating tests, annotating videos, analysing videos to identify key findings and creating research presentations.
Pricing: Basic plan starts at $49 per video for the first 10 videos, and then rises to $99 per video. This will provide you with video and audio of your site or app across a full range of devices, 15 minute maximum video length and a storage limit of 25 videos.
The Pro plan offers a Free trial and quote on request. This allows for a maximum video length of 60 minutes, unlimited video storage, screening and video demographic filters, moderated usability testing, competitive benchmarking, user testing with your own customers, highlight reels, customer experience customer experience analytics and for the research team to summarise key findings.
Finally, UserZoom is an all-in-one SasS customer and usability testing research and analytical solution. They provide a suite of services including recruiting participants for user tests, and research software for mobile and desktop devices, voice of the customer studies, remote usability testing, UX design tools (e.g. card sorting & tree testing) and an online survey tool. In addition they provide support services from defining a study to analysing the data for you.
Pricing: Annual software subscription starts at £19,000 per year. All quotes are customised according your individual requirements and dependent upon the number of user accounts and use of premium features (e.g. UserZoom Recorder and mobile testing capabilities).
Usability research is essential for checking whether a site or app is intuitive and easy to navigate to create a great customer experience. It helps inform our decisions about the choice architecture. Remote usability research solutions or face-to-face user interviews identify the main usability problems. Do these methods of research reflect real behaviour?
How many usability research proposals acknowledge that the process of undertaking usability research can influence the behaviour we observe? We may have taken users out their natural environment and set them objectives that lead them to behave in a certain way.
Behavioural scientists have found that many of our decisions are made automatically by our unconscious brain. The context and our underlying psychological goals heavily influence the choices we make. We also behave differently when we are aware that we are being observed.
Asking respondents direct questions is especially problematic as people over-think issues. They switch to their slow, rational brain when encountering a mentally demanding task. Unfortunately most of the time when we are browsing a website we rely on our fast, intuitive, unconscious brain to make decisions without really engaging our conscious thought process. The implication here is that we cannot even access the rationale behind much of our behaviour when interacting with a website.
Daniel Kahneman, Thinking, fast and slow
“People don’t have reliable insight into their mental processes, so there is no point asking them what they want”.
Avoid taking people away from their natural environment if at all possible. Certainly don’t use focus groups as this is about far away of a normal browsing behaviour as you can get. How often do you search the web with a group of people you have never met and discuss your likes and dislikes of the site?
This is why remote user testing methods have an advantage over some face-to-face methods. Participants can be in their normal environment, with their normal distractions and so their behaviour is less likely to be influenced by the testing process. Don’t get me wrong, there will still be some bias as a result of the testing method. But it may be substantially less than techniques which take the user out of their normal browsing environment.
Observe and listen rather than ask:
You will get more meaningful insights from simply observing and listening to your users during a usability test as past behaviour is a more reliable indicator of future behaviour. Try to avoid verbal interventions as much as possible. People don’t like to admit when they do something wrong and you are likely to influence how they then behave in any future tasks. If you do want some verbal feedback, just ask your testers to say what they are doing as they go through the task.
But always keep in the back of your mind that usability testing is about informing your judgement, and not to prove or disprove someone’s opinions. It is also an iterative process that should begin early on in the development of a design.
Most of our daily choices are made by our fast, intuitive brain which means we don’t have time to rationalise why we are making those decisions. New implicit research techniques such as functional MRI, EEG, biometrics, eye tracking, facial decoding and implicit reaction time studies (IRTs) are allowing marketers to access the sub-conscious part of the brain to better understand how we respond to communications and designs.
Eye tracking research helps identify which specific elements of a page or message attract our attention, but also the communication hierarchy of messages. Heatmaps allows us to display this data to reveal the proportion of visitors who noticed each of the key elements on a page. Plus the frequency and duration of gaze on each element.
Click and mouse movement heatmaps from visual analytics solutions such as Hotjar and Decibel Insights can provide similar insights for existing pages. For true eye tracking research though solutions from Affectiva and Sticky allow for you to evaluate both new and existing web page designs.
In the final analysis the only way you will know if a change identified through usability research improved agreed success metrics is to conduct an online experiment in the form a A/B test. It is only when visitors are acting on their own impulses and with their own money that you will see how they behave.
Prioritise the insights you get from usability testing to decide which are worthy of A/B testing. A/B testing will give you the evidence to show exactly how much difference your usability testing has had on your conversion success metrics.
Online card sorting is a usability tool that helps categorise your webpages by identifying how visitors would expect to find content or functionality. Online card sorting is a quick and simple way of evaluating your information architecture, workflow, menu structure or user navigation journeys. Card sorting tools ask users to organise topics into categories.
Card sorting is sometimes used after a tree testing (or reverse card-sorting) exercise identifies findability problems with current navigation journeys. Tree testing evaluates how easy it is to find an item. It gets participants to solely use the website’s navigation (i.e. without any use of internal search or other navigation aids) to complete a set task. Card sorting using online solutions allows you to quickly identify how customers group topics together.
How Does Online Card Sorting Work?
The card sorting online provider will recruit a sample of people who are roughly representative of your target audience or customer base. Participants organise topics into categories that they feel make sense. They may also label these groups to ensure the words you use are what users would expect.
Online card sorting tools allow you to understand your user’s expectations and their comprehension of your topics. Furthermore, when we discuss our websites we often use jargon and words that are not used outside our organisations to describe aspects of our websites. Knowing how people groups and describe topics can help you:
Organise the structure of your website.
Inform what content to put on your homepage.
Label categories and navigation.
Identify how different groups of users view and organise the same topics.
Limitations of Online Card Sorting:
It does not make allowance for users’ tasks. Card sorting is a content-centric process and if used without considering users’ tasks it can lead to an information structure that is not usable when dealing with real tasks. Make sure you evaluate the output from a card sorting exercise by discussing the potential impact on key user tasks.
It can be superficial as participants may not fully consider what the content is about or how they would use it to complete a task. Card sorting results may also vary widely between participants or they may be fairly consistent. Ensure you don’t rely on too small a sample of users to reduce the risk of a few participants overly influencing your results.
Card sorting online tools should be used to inform your decision making and be viewed alongside other research and usability testing. For example you might want to consider tree testing (reverse card sorting) to evaluate the findability of items in your navigation structure to validate your card sorting findings.
Like any research technique card sorting tools cannot tell you exactly how users will respond on a live website. For this reason it is wise to consider A/B testing any major navigation changes first.
Open and Closed Card Sorting:
Open card sorting involves participants organising topics into groups that make sense to them and then give a name to each that best describes its content. This is great for understanding how users’ group content.
Closed card sorting is where users sort topics using pre-defined categories. This is normally used once you have clearly defined your main navigation or content categories and need to understand how users organise content items into each category.
Often organisations use a combination of the two methods to firstly identify content categories and then to validate how well the category labels work in a closed card sort.
Below I have summarised 8 online (remote) card sorting tools and software solutions for using off-line.
Online Card Sorting Tools:
Online card sorting solutions allow for remote user testing so that you can save on the cost of a lab and it allows participants to conduct the test in the safety of their natural browsing environment.
Remote user testing can also be incredibly quick as participants can be recruited online and asked to complete the study almost immediately.
Here are seven online card sorting tools summarised for you to consider. An additional four offline cards sorting solutions are summarised below the online tools.
Discover how real people think your content should be organised and obtain user insights to make informed decisions about information architecture. Priced at start from $109 per month, $149 per survey or $990 for an annual subscription.
A new online card sorting solution that is in Beta and is currently free for users to try it out. Add or import your cards, add a survey and test your card sort before you launch the exercise. Email your participants a unique URL and you can view results immediately. The UI allows you to analyse data using industry standard tables or download your data to analyse as you wish.
Online card sorting with the ability to turn on subgroups to capture multiple levels of card placement. This PRO feature allows users to drag one grouping of cards into another grouping. An additional PRO feature offers participant replay which logs every decision made by users and logs each time they sort a card, create a new group or rename an existing group.
Free demo-account allows you to try out the service with a simple card sort. A Basic subscription starts at $49 for 30 days or $99 for the Pro 30 day plan.
A full-service supplier of usability research, Usability Sciences has been established for over 25 years and will design, manage and analyse the result of your card sorting research for you. They offer both open and closed card-sorting solutions for you.
Card sorting is just one of the tools in their impressive UX suite. Supports open and closed card-sorts, and randomisation of cards and categories. Offers a 14 day Free trial and you can obtain a price quote by submitting your details using a short form.
Offers clients a full usability suite, including web-based card sorting. Supports up to 100 items and 12 categories. Supports open and closed card-sorts, randomisation of questions to reduce participant bias, and follows a responsive design so participants can take studies on either their desktop or iPad. Using an iPad makes the process more of an intuitive experience by harnessing the power of touch-screen technology.
UserZoom is Ideal if you are a large organisation looking for a comprehensive usability testing programme, including information architecture/UX design, benchmarking and market research. For businesses subscriptions start from $19,000 a year.
Off-Line Tools:
If you prefer to conduct card sorting offline with users you have recruited locally there are a number of free software solutions available to use. Summarised below are four free card sorting software tools you can use.
This is a free open-sourced card sorting software that you can download onto a computer running Windows (Windows 7 is preferred). It allows you to import a list of cards using Word or Excel and the software enables you to sort up to 1,000 cards. Users click and drag cards into pre-set categories and you can view results using real-time reporting.
Data can be exported and merged, with each participant’s data presented and exported individually. The software does allow you to aggregate results and run a cluster analysis. The reporting uses a dendrogram or family tree to present results.
However, the software is not for everyone as it requires basic knowledge of SQL databases because installation includes SQL Compact. However, the site does provide a step-by-step installation guide to help you complete the process.
This is a free card sorting tools designed for Macintosh. The tool offers both open and closed sorting, plus a combination of the two. The software simulates a table with cards on it so that participants can click and drop cards into the relevant category. It also allows respondents to create sub-categories if needed which can be useful for developing drop down menus.
The reporting functionality allows you to view results in real-time and export data if required. The developers plan a web-based version of the software to allow participants to respond to studies without having to be on a Macintosh.
Finally:
Many of these online card sorting tools offer a free trial or demo so don’t let cost put you off trying out card sorting to improve your information architecture. This is such important element of the user experience don’t leave it all to chance. Get some input from real users. You should also seriously consider using tree testing to validate card sorting results and evaluate the findability of categories or products on your site.
2. Loop11:
, which I highly recommend you read. He advocates doing your own usability research if you can. Undoubtedly this is a great idea if you have the time and the equipment. If you have no or very little budget you might as well do this as even one user test is better than none.
