A definition of usability from ISO 9241-11:
[...] the extent to which a product can be used by specified users to achieve specific goals with effectiveness, efficiency and satisfaction in a specified context of use.
12.2 Designing a user study
12.2.1 Dependent vs. independent variables
Dependent variable: what you measure
- task completion time
- number of errors
- satisfaction rating
Independent variable: what you manipulate (what you compare)
- prototype A vs. prototype B
- gender: men vs. women
- level of expertise: novice vs. intermediate vs. expert
12.2.2 Types of data
- Nominal data
- unorderend categories (apple, banana, orange...)
- Ordinal data
- ordered categories (low, medium, high)
- Interval data
- distances meaningful (temperature, distance...)
12.2.3 Within-subjects vs. between-subjects
- subject X‘s performance on P1, P2
- subject X compares versions A, B, ...
- group X performs on P1, group Y on P2
- groups X, Y, ... rate versions A, B, ...
Problem: Order of task performance can impact results
- usually because of increasing experience
- can it be that task X is performed such because it occurred always at position 2 ?
- every task must be at every position the same number of times
- NB: more constraints may be necessary => pseudo-random orders