The information architecture of a site or app is crucial to creating a good user experience. A good test of the information architecture is to evaluate how easy it is for visitors to find what they are looking for. However, many websites and apps have never tested the findability of items in their navigation structure (often called taxonomy) with real users.
This is a concern as how do you know whether your navigation structure is intuitive and easy to use unless you test it? Tree testing or reverse card sorting is a technique that can significantly reduce problems with an important element of your information architecture, your navigation structure.
What is Tree Testing?
Source: Freeimages.com
Tree testing evaluates the findability, labelling and organization of topics on a website. Most websites are organised into a hierarchy (a “tree”) of topics and subtopics. Tree testing is a way of identifying how easy it is for users to find individual items in this hierarchy to improve the information architecture.
However, unlike normal usability testing, tree testing is not carried out on the website itself. Instead users browse a simplified text version of the site structure. This removes the effects of the design, including visual cues, and navigational aids (e.g the internal search box) and other factors that might influence how quickly visitors find what they are looking for.
How Does Tree Testing work?
There are 6 steps to complete a tree test:
Users are given a find it task to complete (e.g. “find a portable DVD players for less than £20).
Participants are shown a text list of the top-level topics of the website.
Users select a heading, and then are given a list of the subtopics to choose from.
Participants continue choosing topics in the tree. They can backtrack if necessary, until they find a topic that achieves their aim or they may abandon the process if they can’t find what are looking for.
Users will then repeat the process a number of times with different find it tasks to test the findability of a range of items in the tree hierarchy.
Test results will then be analysed once a sufficient number of users have completed the test.
Example of welcome screen for remote tree testing – Source: Userzoom.com
When Should You Use Tree Testing?
If you want to identify the root cause of navigation problems tree testing may be the best solution. It removes the effect of the design of your website and other navigational tools and aids from the equation. With no internal search to assist your user tree testing helps to isolate navigational deficiencies so that you can make the necessary improvements in your taxonomy. Tree testing is often used for:
Identifying which items, groups or labels are causing problems for your users and set a benchmark of “findability” before you update your navigation. This might then lead you to conduct a card sorting exercise to improve the usability of your taxonomy.
Measuring the impact of an improvement or change in the findability of items in your navigation structure. This will allow you to validate if the change you are making helps improve findability, makes no difference or actually creates a new problem.
Which Elements should You Test?
For a small website with less than a hundred items you may be able to test your whole navigational structure. However, for large ecommerce websites with literally thousands of items on the site this is not practical or cost effective. In this instance you should use your web analytics to identify less common paths that can be removed from the testing process.
To decide what to test you should start by defining user’s goals and the top tasks that they need to accomplish to meet their goals. This normally involves getting both users and stakeholders to rank the main tasks so that you can identify what both groups agree on. They also identify any low priority tasks that internal stakeholders wrongly believe are important to users. It may be useful to include some items that cross departments as these create their own issues for users and items that have been identified as problematic from open card sorting or Voice of Customer research.
What Sample Size Do You Need?
As Steve Krug points out, “Testing one user is 100% better than testing none.” Whilst this is true, we have to bear in mind that with tree testing we may be dealing with a complex navigation structure. It is important to conduct a reasonably robust test if we are to draw any reliable conclusions. The key outcome metric should be whether the user successfully found the item they were asked to locate. This simplifies the analysis to a “Yes/No” metric.
I have outlined below the sample size required to achieve a confidence level of 95% and assumed 50% of users find the item. I have assumed this because 50% generates the highest possible margin of error and so is the worst case scenario.
Sample size required for specific margin of error at 95% confidence level.
Generally you should limit the number of tests each participant completes to 10 depending upon how long on average each task takes to complete. Otherwise participants may become fatigued and they will also become experienced users of your site structure which could influence the test results.
Should You Ask Participants Questions?
After each tree test it is useful to ask participants to rate the difficulty of the task. This can provide a guide to the usability of finding the item. Keep questions to a minimum but understanding how users perceive a task can add context to the test data. It can be useful for instance to compare task completion data with survey answers to identify any items where user perception does not align with task completion. This could highlight areas of particular concern.
3 Remote Tree Testing Solutions:
Tree testing may not be one of the most well-known forms of usability research. But it certainly offers the potential to help organisations resolve problems with their navigation structure and improve the overall information architecture. If you want to investigate tree testing further you can check out these solutions:
One of the leaders in web-based usability testing for information architecture. Treejack is a popular solution for evaluating website navigation without the normal visual distractions.
Offers a web-based solution and will analyse the findings to determine the effectiveness of your site structure. They will provide specific recommendations on changes to your labels, structure and placement of content within your navigation hierarchy.
Provides a web-based service to identify navigational issues early in the design process. UserZoom will analyse any attempts where participants have trouble navigating to ensure this is resolved before your site goes live. It will also give you a measure how well users can find items in your hierarchy.
To create a pain free user experience and optimise conversion it is essential to carry out usability testing research. Sure, you can ask a few people around the office to check out your new site, but it is more important to get feedback from ‘real users’ who do not work in e-commerce and are not connected to your business.
Psychology tells us that as we concentrate on a task or project we are prone to see what we expect to see because our visual cortex unconsciously takes the decision to filter out things that it regards as less important to achieving a task. This is why we often miss the most obvious mistakes if we proof read our own work. Usability testing removes this filter as it does not rely on our own opinions.
IKEA Effect
We also get too close to our pet projects and as a result we overvalue the things we create, (see the IKEA effect). As a result we are not the best people to evaluate websites that we helped to create. Anyone connected to your business may also suffer from some of the same cognitive biases or may just not want to hurt your feelings. Usability testing avoids these biases by getting the views of ‘real users’, that is people not connected with your organisation.
Source: Freeimages.com
Usability testing is not about proving or disproving something works or not. It is about informing decisions and giving you insights into how users interact with your website. If you need a definitive answer then you really should be conducting an A/B test. As I pointed out in another post on whether usability research is reflecting real behaviour all research is subject to bias and limitations.
People for instance change their behaviour when they are aware they are being observed. This can be a particular problem for usability testing which is conducted offline. However, it is possible to build procedures into usability testing to minimise this problem. For example you can get the user to complete a few fake tasks to disguise the real test experience.
When Should You test?
Source: Freeimages.com
The earlier you do some usability testing the better as this will allow you to respond to user feedback at each step in the development and design process. Wire frames, prototypes or even drawings can be tested to give you useful feedback before you move onto finished designs.
Don’t use focus groups as usability research needs to deal with one user at a time. Otherwise people can get distracted by what other people are doing and you also need to give them your full attention. It is important that you observe and listen to users and avoid asking questions as people will over-think their behaviour if asked to explain it.
How Should You Test?
Source: Freeimages.com
Steve Krug has written an awesome book, Don’t Make Me Think: A Common Sense Approach to Web Usability (Voices That Matter), which I highly recommend you read. He advocates doing your own usability research if you can. Undoubtedly this is a great idea if you have the time and the equipment. If you have no or very little budget you might as well do this as even one user test is better than none.
However, there are also some affordable online services available, some of which are free. The benefit here is that they can manage all the admin and recruitment for you, plus conduct the usability testing and if required do the analysis. All you have to do is agree the brief. I would still recommend you get videos of your usability tests as you will often learn more from watching people browse your site than from reading a report. A video brings it to life in a way a report cannot.
One other advantage is that such suppliers offer a range of solutions including remote automated usability testing, card sorting for developing your navigation, tree testing to evaluate how easy it is find content on a site and eye-tracking to identify where on page attention is drawn to. Such suppliers can also use their expertise to advise you on how to best design a usability test study.
Who Should You Recruit?
Source: Freeimages.com
Some usability test solutions, such as Hotjar, offer you the ability to serve pop-ups on your site to recruit your own visitors to complete specific tasks. You can then share screens using Skype or other web meeting tools to observe how successful they are at achieving the set task. Alternatively many suppliers offer the option to recruit testers who match your user demographics.
Don’t get too obsessed though with matching your target audience as usability testing is about understanding how people in general interact with your site. Setting very strict recruitment criteria will just increases the cost and time needed to conduct the testing without adding much value to the outcome.
Online usability testing with your first project free (up to 5 tasks and 2 questions). Covers over 40 languages, provides heatmaps and clickstream analysis, real-time reporting, and you can test on mobile devices.
Pricing: A Free usability test is available for new customers. Pay as you go costs $350 per project. All plans include 1,000 participants per project, unlimited tasks and questions, testing on mobile, real-time results and 24/7 email support.
The Micro plan costs $158 a month and is designed for organisations with between 1 and 10 employees, plus for non-profits and public sector clients. The SMB plan costs $410 per month and is for 11 to 100 employees. The Enterprise plan is priced at $825 per month.
Remote usability testing which provides videos of visitors undertaking set tasks on your website. You also get written answers to questions you set. Get your first test for Free – normally costs $35.
Pricing: The Personal plan charges $35 per test credit. A desktop test requires 1 credit, whilst a mobile test costs 2 credits. Includes up to 20 minutes of video and audio feedback, written responses to custom survey questions and the ability to analyse your results with tagged, time stamped annotations.
The Team plan costs $299 per month. This gives you 10 credits per month, testing with your own users for one month, multi-user login, collaborative video annotation, crowd sourced key insights with the UXCrowd, UX diagnostics and the ability to download your video results and test data.
The Enterprise plan is not priced on the website. However, this
includes 100 test credits per month, unlimited testing with your own users, extended 30-minute length for test results and one-click report generation integrated with video playback.
UsabilityHub have a great selection of simple but effective usability testing solutions. You can obtain first impressions of your mock-ups and designs, see where visitors want to click or discover how easy visitors find it to navigate your website.
Simply upload an image, and select the type of test you’d like to run.
You can choose from:
Five Second Test to understand people’s first impressions of
your design.
Click Test to find out where they click and how they interact
with your interface
Navigation flow test to identify how visitors navigate around your
website or applications.
Question Test – allows you to conduct fast surveys by uploading an image and asking users questions about the design.
You can then decide how many people you want to be in the test or even recruit your own testers. UsabilityHub then create a report showing a detailed breakdown of the interactions each tester had with your design.
Pricing: Responses from testers you recruit are free. Testers recruited by UsabilityHub cost 1 credit each and responses from testers of specific demographics cost 3 credits each.
The Free Community plan allows you to create unlimited tests, with responses from your own users being free and buy responses from UsabilityHub from $1 each.
The UsabilityHub Pro plan costs $99 a month and allows you to buy responses at 50% off all credit purchases, starting at just 50 cents per response. Create unlimited tests, customize the test experience with messaging and redirection after the test, use a single link for multiple tests in a row and target particular demographics.
Established over 25 years ago Usability Sciences offers a full managed service for usability testing, offering a comprehensive range of solutions including card sorting, rapid iterative testing, mobile & tablet user testing and eye-tracking research.
Watch videos of real users talking about what they think as they use your website. UserBob recruits people to visit your website. Set a scenario for the user and specify a task for them to attempt to complete. The user then goes to your website and tries to complete your task. During their visit they record their screen and voice as they think out loud about the experience. You then receive a copy of the video to learn about what users say about your site.
You decide how many users you need, what demographics match your visitors, and how long each one should spend on your website. The test is instantly made available for users to participate and you will normally have a video to review within a few hours.
Pricing: Start at just $10 for First Impressions where 10 users will spend one minute each on your website. Users will discuss their first impressions of your website, who they think it is for and what you can do on the site. Task Completion costs $20 for 5 users who spend 4 minutes attempting to complete your task. The price of the Custom test is variable. This involves between 1 to 10 users each spending up to 8 minutes with a specific scenario and user task to complete. You may also specify user demographics for Custom tests.
Get 5 to 15 minute videos of users recording their experience on your site and identify where the pain points are. Hear what users are thinking on their personal devices and in their natural environment.
Omni-channel usability testing solution. Will supply user testers from their panel or recruit to your specific demographic requirements. Alternatively you can recruit participants using a customisable invitation widget, by posting a link on blogs, websites, twitter, by using TaskRabbit, Mechanical Turk, Craiglist or by using a third party panel provider.
Userlytics allows you to test prototypes, videos, mobile apps, display ads, search and social behaviour, desktop and web applications, smart-phones and tablets and websites.
Pricing: Starts from $49 per user tester and depends upon the testing features you require, whether you need respondents recruiting, demographic needs, session length and reporting requirements.
Get videos in an hour of real people speaking their thoughts as they use your website, apps, prototypes and more.Do it yourself or access User Testing’s on-demand panel of over one million users to find an exact match of your target audience.
You can either select your users and write your own tasks or use the expect research team to complete such tasks as creating and managing tests, long term research road-mapping, moderating tests, annotating videos, analysing videos to identify key findings and creating research presentations.
Pricing: Basic plan starts at $49 per video for the first 10 videos, and then rises to $99 per video. This will provide you with video and audio of your site or app across a full range of devices, 15 minute maximum video length and a storage limit of 25 videos.
The Pro plan offers a Free trial and quote on request. This allows for a maximum video length of 60 minutes, unlimited video storage, screening and video demographic filters, moderated usability testing, competitive benchmarking, user testing with your own customers, highlight reels, customer experience customer experience analytics and for the research team to summarise key findings.
Finally, UserZoom is an all-in-one SasS customer and usability testing research and analytical solution. They provide a suite of services including recruiting participants for user tests, and research software for mobile and desktop devices, voice of the customer studies, remote usability testing, UX design tools (e.g. card sorting & tree testing) and an online survey tool. In addition they provide support services from defining a study to analysing the data for you.
Pricing: Annual software subscription starts at £19,000 per year. All quotes are customised according your individual requirements and dependent upon the number of user accounts and use of premium features (e.g. UserZoom Recorder and mobile testing capabilities).
, which I highly recommend you read. He advocates doing your own usability research if you can. Undoubtedly this is a great idea if you have the time and the equipment. If you have no or very little budget you might as well do this as even one user test is better than none.
