Synopses & Reviews
One very important aspect of usability is being able to measure how usable a system really is, and how changes to the user interface of the system impact the end-user experience. This aspect of usability is commonly known as usability metrics. Usability metrics are essential if the researcher wants to know how to improve the usability of any interactive system, what aspect of the system needs improvement, the priority of usability problems, and whether the usability goals have been achieved. There are many different ways to measure usability, each way having distinct advantages and limitations. The proposed book would be the first comprehensive view on how to collect, analyze, and present usability metrics.
Many different types of usability metrics will be discussed in the book. The metrics are organized into the following six general categories:
?Performance metrics focus on how effectively and efficiently users perform tasks. Specific performance metrics covered will include success, completion time, errors, learnability, and other efficiency metrics such as click through rates and abandonment rates.
?Issues-based metrics are based on the identification of specific usability issues and the severity of those issues. These are the most common type of metrics used by nearly all usability specialists.
?Self-reported metrics relate to what an end-user reports about their experience. Examples of self-reported metrics include satisfaction, ease of use, usefulness, awareness, and expectations.
?Web Navigation metrics are based on how users navigate through a web site, such as click-through rates, abandonment rates, stickiness, and optimal paths.
?Derived metricssummarize the usability of more than one task or using more than one type of metric. Derived metrics include single usability scores and usability score cards. Derived metrics provide a valuable, higher-level view of an entire interface.
?Physiological metrics focus on phenomena which can be directly or indirectly measured from the human body. Physiological metrics may indicate the usability of a system through behaviors such as eye-movements, stress, and facial expressions.
* First, long-awaited book treatment of practical usability measurement, which enables usability professionals to develop a broad understanding of usability metrics;
* Provides a comprehensive guide for choosing the most appropriate metrics, with the advantages and limitations of each metric, with examples of how different metrics are used;
* Detailed information on how to use each metric: how to collect data in a time and cost efficient way, most common ways to analyze data, and which method is most appropriate for the reader's goals.
* Detailed case studies and examples help teach process and technique.
The first practical, detailed instructions on how to measure improved usability for the web, web-based applications, and electronic products of all kinds.
"If Tom and Bill could convince me, perhaps the world’s biggest fan of qualitative testing, that usability metrics are really valuable—which they have, in this wonderful book—then there’s no doubt they’ll convince you. I loved reading this book, because it was exactly like having a fascinating conversation with a very smart, very seasoned, and very articulate practitioner. They tell you everything you need to know (and no more) about all the most useful usability metrics, explain the pros and cons of each one (with remarkable clarity and economy), and then reveal exactly how they actually use them after years and years of real world experience. Invaluable!" Steve Krug, author of Don’t Make Me Think: A Common Sense Approach to Web Usability
"This book is a great resource about the many ways you can gather usability metrics without busting your budget. If you’re ready to take your user experience career to the next level of professionalism, Tullis and Albert are here for you and share generously of their vast experience. Highly recommended."
Jakob Nielsen, Principal, Nielsen Norman Group, author of Usability Engineering and Eyetracking Web Usability
"If you do any type of usability testing, you need this book. Tullis and Albert have written a clear and comprehensive guide with a common-sense approach to usability metrics."
Ginny Redish, President of Redish and Associates, Inc., author of Letting Go of the Words
Effectively measuring the usability of any product requires choosing the right metric, applying it, and effectively using the information it reveals. Measuring the User Experience provides the first single source of practical information to enable usability professionals and product developers to do just that. Authors Tullis and Albert organize dozens of metrics into six categories: performance, issues-based, self-reported, web navigation, derived, and behavioral/physiological. They explore each metric, considering best methods for collecting, analyzing, and presenting the data. They provide step-by-step guidance for measuring the usability of any type of product using any type of technology.
• Presents criteria for selecting the most appropriate metric for every case
• Takes a product and technology neutral approach
• Presents in-depth case studies to show how organizations have successfully used the metrics and the information they revealed
About the Author
Tom Tullis is Vice President of Usability and User Insight at Fidelity Investments and Adjunct Professor at Bentley University in the Human Factors in Information Design program. He joined Fidelity in 1993 and was instrumental in the development of the company’s usability department, including a state-of-the-art Usability Lab. Prior to joining Fidelity, he held positions at Canon Information Systems, McDonnell Douglas, Unisys Corporation, and Bell Laboratories. He and Fidelity’s usability team have been featured in a number of publications, including Newsweek , Business 2.0 , Money , The Boston Globe , The Wall Street Journal , and The New York Times.Bill Albert is Director of the Design and Usability Center at Bentley University. Prior to joining Bentley, Bill was Director of User Experience at Fidelity Investments, Senior User Interface Researcher at Lycos, and Post-Doctoral Research Scientist at Nissan Cambridge Basic Research. Bill is an Adjunct Professor in Human Factors in Information Design at Bentley University and a frequent instructor at the International Usability Professional’s Association Annual Conference. Bill has published and presented his research at more than thirty national and international conferences. He is coauthor (with Tom Tullis) of Measuring the User Experience and Beyond the Usability Lab. He is on the editorial board for the Journal of Usability Studies.
Director, Design and Usability Center, Bentley University, USA
Table of Contents
Introduction; Background: Data Types; Sampling Size; Experimental Design; Data Analysis. Overview of Usability Metrics: Types of Metrics; Methods and Metrics; Summative vs. Formative; Choosing Appropriate Metrics. Performance Metrics: Task Success; Completion Time; Errors; Efficiency (clicks, pages, steps, etc.). Issues-Based Metrics: What is a Usability Issue; Severity Ratings; Test Biases; Reporting Positive Issues. Peferential-Based Metrics: Satisfaction; Ease of Use, Usefulness; Expectations; Standard Questionnaires. Web Navigation Metrics: Web-page Click-through Rates; Web page Abandonment Rates. Derived Metrics: Task-based; Aggregate. Observational Metrics: Eye Movements; Stress; Facial Expressions; Other Observational Metrics. Case Studies. Special Topics: Six Sigma and Usability; Automated Methods; Discount Techniques; Server Log Analysis; A/B Testing. Conclusion: Communication to Management; Cost Justification; Industry Trends.