Lecture 5: User Testing
Overview
This lecture provides a comprehensive introduction to User Testing, a crucial process where actual users interact with a system or product to identify usability issues and pain points. We'll explore the foundations laid by CHI pioneers like Jakob Nielsen, Donald Norman, and Ben Shneiderman, and examine various testing methodologies used in modern software development.
Lecture Material (PDF)
CHI Pioneers & Foundations
The field of Human-Computer Interaction (CHI) was shaped by influential researchers who established the principles we use today. Understanding their contributions provides context for modern user testing practices.
Jakob Nielsen
Advocated for testing with small groups and popularized the 'think aloud' method. Proposed 10 usability heuristics used alongside user testing to identify design flaws.
Donald Norman
Introduced the concept of affordances - perceived properties of objects that suggest how they should be used. Stressed the importance of clear, immediate feedback to users.
Ben Shneiderman
Developed the Eight Golden Rules of interface design including 'Strive for consistency' and 'Seek universal usability'. Advocated for direct manipulation interfaces.
Early CHI Research
Laid the foundation for modern usability engineering through rigorous user testing. This is empirical, largely quantitative, and what we call a 'discount' technique.
User Testing Methods
Various testing methods provide different insights into usability. Combining these approaches creates a comprehensive understanding of how users interact with your product.
Think Aloud Protocol
Users verbalize their thoughts while using the system to identify usability challenges. Reveals the user's mental model and decision-making process in real-time.
Task-Based Testing
Testing based on real-world tasks that users would perform to simulate actual usage scenarios. Identifies pain points in specific user journeys.
A/B Testing
Two versions of a design are compared to see which performs better with real users. Provides quantitative data on design effectiveness.
Cognitive Walkthrough
Stepping through the interface as a user would, predicting usability challenges before testing. Useful for early-stage evaluation without user involvement.
Paper Prototyping
Using paper prototypes for testing designs cheaply and effectively before digital implementation. Bill Buxton has written extensively on this topic.
Remote User Testing
With remote tools, user testing can be conducted anywhere, providing a broader and more diverse set of feedback from geographically distributed users.
Nielsen's 10 Usability Heuristics
Jakob Nielsen proposed 10 usability heuristics that can be used alongside user testing to identify design flaws. These principles guide both evaluation and design decisions.
Visibility of System Status
The system should always keep users informed about what is going on through appropriate feedback within reasonable time.
Match System & Real World
The system should speak the users' language with familiar words, phrases, and concepts rather than system-oriented terms.
User Control & Freedom
Users often choose system functions by mistake and need a clearly marked 'emergency exit' to leave the unwanted state.
Consistency & Standards
Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform conventions.
Error Prevention
Testing can identify areas where users are prone to making errors. Good design prevents problems from occurring in the first place.
Recognition Over Recall
Minimize the user's memory load by making objects, actions, and options visible. Instructions should be visible or easily retrievable.
Usability Metrics & Measurement
Early CHI research emphasized the importance of measuring usability with quantifiable metrics. These help evaluate the design's effectiveness objectively.
Task Completion Rate
The percentage of users who successfully complete a given task. A fundamental metric for measuring usability success.
Time-on-Task
How long it takes users to complete specific tasks. Shorter times generally indicate better usability, though context matters.
Error Rate
The number of errors users make when completing tasks. Helps identify confusing or problematic interface elements.
Learnability
How quickly a user can become proficient in using a new system. Nielsen stressed this as a key usability attribute.
Memorability
When users return to the system after a period of not using it, how easily can they remember how to use it?
User Satisfaction
Subjective measures of how pleasant the interface is to use. Surveys provide direct user feedback on specific aspects.
Advanced Testing Considerations
Beyond basic usability testing, several advanced topics ensure comprehensive coverage of user experience across different contexts and user groups.
User Personas
Creating user personas guides testing by representing real-world user groups and ensuring their needs are met throughout the design process.
Mental Models
Don Norman's work showed that users build mental models of how systems should work. Testing helps ensure the design aligns with those models.
Affordances
The perceived properties of an object that suggest how it should be used. Testing ensures these are clear and intuitive to users.
Cultural Considerations
Testing with users from different cultural backgrounds can reveal usability issues not apparent with a homogenous testing group.
Accessibility Testing
User testing is crucial for identifying accessibility issues and ensuring products are usable by people with varying abilities.
Real-World Contexts
Testing in real-world environments through field studies can uncover usability issues that controlled lab testing might miss.
Mobile Usability
Mobile devices introduced new usability challenges. CHI researchers were among the first to highlight the importance of mobile user testing.
Multimodal Interfaces
CHI research explores usability of interfaces where users interact via touch, voice, and other inputs. Fitts' Law provides empirical evidence for these mechanisms.
Tools & Further Reading
VLE Page
Course management and resources.