What a prototype can and cannot tell you: fidelity, paper and the test

Every studio brief ends with the same question: how high-fidelity does this prototype need to be? It sounds practical, and it is the wrong question. A prototype is a claim about future use. What makes it testable is the tasks the participant is given, and who the participant is, not the finish on the artefact. This unit makes three claims. Fidelity is not what makes a test informative. Low-fidelity artefacts stay valid for finding usability problems. Practice has moved to digital-first tools for reasons of collaboration rather than evidence.

The argument matters because the paper prototype, the studio's cheapest habit, is the first thing a team abandons when a deadline tightens. Fidelity barely affects what a test finds. That drift is therefore working conditions wearing the clothes of a methodological advance. This is the sixth of ten units; it takes one chosen idea and hands unit seven the question of whom the design is for.

What a prototype is for, not what it is made of

Houde and Hill (1997) offered the reframing. They do not describe a prototype by its attributes. They describe it by what it prototypes. Its role is the function the artefact serves in a user's life. Its look and feel is the concrete sensory experience of using it. Its implementation is the techniques and components that make it work. Each raises a different class of question, and one prototype can answer one while saying nothing about the others.

McCurdy, Connors, Pyrzak, Kanefsky and Vera (2006) turned the insight into five dimensions for planning a prototype: visual refinement, breadth of functionality, depth of functionality, richness of the data, and richness of the interaction. Their case is a tool for NASA Mars mission planning, built high on the dimensions their own questions needed and low on the rest, and the data it produced predicted how users would later perform with the real system. That is the unit's analytical tool. Once fidelity is a set of choices, one question becomes several.

Fidelity is a weak lever on which problems a test surfaces

The empirical case is older than current practice suggests. Walker, Takayama and Landay (2002) tested sketched and HTML versions of the same website prototypes in paper and on a computer. They separated the medium from the level of fidelity. Low- and high-fidelity prototypes were equally good at uncovering usability issues, and the results did not depend on the medium.

Virzi, Sokolov and Karis (1996) reached the same conclusion with two products, a CD-ROM electronic book and an interactive voice response system. In both experiments the two conditions found substantially the same sets of problems. The proportion of participants detecting a given problem correlated between them. Sefelin, Tscheligi and Giller (2003) asked whether the medium changed people's willingness to criticise a design. It was largely unrelated.

Three studies, two products: fidelity is a weak lever on which problems a test surfaces. The task and the participant do the work.

Where fidelity does change what a test finds

The claim is not that fidelity is irrelevant. Lim, Pangam, Periyasami and Aneja (2006) compared paper-based, computer-based and fully functional prototypes of a mobile phone's text messaging. They found that each prototype's characteristics shaped the evaluation differently, and that some problem types depended on fidelity. Equivalence is a tendency, not a licence to ignore the artefact.

Sauer and Sonderegger (2009) varied fidelity and visual appeal independently with 60 participants: completion time may be overestimated with a computer prototype, and users appeared to compensate for deficiencies in aesthetic design by overrating the aesthetic qualities of reduced-fidelity prototypes. Sauer, Franke and Ruttinger (2008) ran the applied version. In it, 30 users chose settings on a paper prototype of a washer's temperature control, and 34 operated the real appliance. Both compared standard with information-enriched labelling. Enriched labelling helped in both. The comparison suggested, however, that paper prototypes may overestimate a design change's behavioural effect.

The qualification is narrower than the folklore. Fidelity rarely decides whether a problem is found. It can decide how large an effect looks, so a low-fidelity test supports a judgement about presence better than about size.

The paper method turns the unfinished look into evidence about behaviour

Rettig (1994) gives the canonical account. Build the interface on paper, give real users real tasks, and have one member of the team play the computer, moving the paper parts in response. He argues that the unfinished look turns feedback from appearance towards behaviour. He reports a workshop in which four teams learned the technique, built a menu prototype, tested it and measurably improved the design in about six hours.

Low fidelity is not confined to screens. Dahlback, Jonsson and Ahrenberg (1993) set out the Wizard of Oz method, in which a person simulates the system's responses, so interaction can be studied before the system exists. Buchenau and Suri (2000) describe experience prototyping, in which designers, users and clients engage with a simulation to appreciate a future condition.

Practice has moved, and the reason is not the evidence

Chen and Yoon (2024) interviewed ten practitioners and educators, and found a decisive move towards digital-first prototyping. Their informants cite three reasons. Hybrid work means a shared file travels where paper cannot. All-in-one tools make high fidelity cheap. Corporate preference favours prototypes that look finished. Low-fidelity prototyping is losing ground for reasons of collaboration, tooling and culture.

What a test establishes has been reframed in parallel. Sauer, Sonderegger and Schmutz (2020) review the definitions and measures of usability, user experience and accessibility. They argue for integrating them as interaction experience, reported through spider charts rather than a single score. A prototype evaluation now carries more questions than a problem list answers. Accessibility belongs in that set from the start: a test whose participants all see, hear and use a fast connection cannot report on a design's fit for anyone else.

Contested question: does low fidelity stay valid as the artefact grows richer?

The literature divides on what a low-fidelity test can be trusted to show, and both camps read the same studies.

One position holds that fidelity is not the driver. Walker, Takayama and Landay (2002) found low and high fidelity equally good at uncovering usability issues, and independent of medium. Virzi, Sokolov and Karis (1996) found the same problem sets across two products. Sefelin, Tscheligi and Giller (2003) found the medium largely unrelated to users' willingness to criticise. On this reading the drift costs the evidence base nothing.

The other holds that fidelity distorts. Lim and colleagues (2006) found the prototype's characteristics shaping which problems appeared. Sauer and Sonderegger (2009) found completion time overestimated with a computer prototype. Sauer, Franke and Ruttinger (2008) found paper prototypes overestimating a design change's behavioural effect. On this reading a low-fidelity test screens rather than measures.

Both survive because they measure different things: the equivalence studies count problems, while the distortion studies measure magnitudes and affect. Settling the first would need a study that varied one named fidelity dimension at a time on a standard task, holding participants constant and reporting both the problems found and an efficiency measure. The second needs a comparison of a digital-first and a paper prototype of the same content, with the same tasks and participants, since Chen and Yoon (2024) measure no outcomes.

The two sides read the same studies and measure different things. The equivalence studies count which problems appear, across three studies and two products, which is the firmer evidence, while the distortion studies measure how large an effect looks and how a prototype feels, and find bias in magnitude rather than in presence. The strongest objection is that equivalence may hold only because these tasks throw up coarse problems: a test can list the same problems at either fidelity while still mis-ranking which one matters most. While the dispute is open, a designer uses a low-fidelity prototype to establish that a problem exists, and reserves a high-fidelity artefact for judging how much it costs.

Where the argument is heading

Three directions are visible in named work. McCurdy and colleagues (2006) replaced one fidelity scale with five, and later work treats the combination as the design decision. Sauer, Sonderegger and Schmutz (2020) widen what an evaluation reports, moving from a problem count to usability, experience and accessibility together. Chen and Yoon (2024) show a profession re-tooling around hybrid work while the equivalence evidence stands unchanged. A conditional rule is emerging: not which fidelity, but which dimension, for which question, tested with whom. That sets up unit seven, where the prototype's user becomes the thing designed for.

The sources this unit rests on

• Buchenau, M. and Suri, J. F. (2000). Experience prototyping. Proceedings of the 3rd Conference on Designing Interactive Systems (DIS). doi:10.1145/347642.347802 Buchenau and Suri define experience prototyping as engaging with a prototype to appreciate an existing or future condition first-hand, illustrated from commercial projects. They measure no outcome, and the paper extends the prototype's job beyond layout to experience.
• Chen, K. and Yoon, J. (2024). Exploring the diminishing allure of paper and low-fidelity prototyping among designers in the software industry. Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI). doi:10.1145/3613904.3642774 Chen and Yoon interviewed ten practitioners and educators and found a move to digital-first prototyping driven by hybrid work, all-in-one tools and corporate preference for polished screens. It evidences practice, not effectiveness.
• Dahlback, N., Jonsson, A. and Ahrenberg, L. (1993). Wizard of Oz studies: why and how. Knowledge-Based Systems. doi:10.1016/0950-7051(93)90017-N Dahlback, Jonsson and Ahrenberg set out the Wizard of Oz method, in which a person simulates the system's responses, arguing that dialogue systems need empirical study of the man-machine situation. It extends low-fidelity testing from layout to behaviour that cannot yet be built.
• Houde, S. and Hill, C. (1997). What do prototypes prototype?. Handbook of Human-Computer Interaction, Elsevier. doi:10.1016/B978-044481862-1.50082-0 Houde and Hill propose describing a prototype by what it prototypes: role, the function it serves in a user's life; look and feel, the concrete sensory experience; and implementation, the techniques that make it work. The chapter replaces one fidelity scale with the question a prototype answers.
• Lim, Y. K., Pangam, A., Periyasami, S. and Aneja, S. (2006). Comparative analysis of high- and low-fidelity prototypes for more valid usability evaluations of mobile devices. Proceedings of the 4th Nordic Conference on Human-Computer Interaction (NordiCHI). doi:10.1145/1182475.1182506 Lim, Pangam, Periyasami and Aneja compared paper-based, computer-based and fully functional prototypes of a mobile phone's text messaging. Each shaped the evaluation differently, and they identify what causes the differences in the problems found.
• McCurdy, M., Connors, C., Pyrzak, G., Kanefsky, B. and Vera, A. (2006). Breaking the fidelity barrier. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI). doi:10.1145/1124772.1124959 McCurdy and colleagues set out five dimensions along which prototypes can be planned, then built a mixed-fidelity prototype for NASA Mars mission planning tools, high on some dimensions and low on others. It predicted eventual user performance.
• Rettig, M. (1994). Prototyping for tiny fingers. Communications of the ACM. doi:10.1145/175276.175288 Rettig gives the canonical account of low-fidelity prototyping: build the interface on paper, give real users real tasks, and have one team member play the computer. He reports a workshop in which four teams learned the method, built and tested a menu prototype and measurably improved the design in about six hours.
• Sauer, J., Franke, H. and Ruttinger, B. (2008). Designing interactive consumer products: utility of paper prototypes and effectiveness of enhanced control labelling. Applied Ergonomics. doi:10.1016/j.apergo.2007.03.001 In two studies, 30 users chose settings on a paper prototype of a washer's temperature control and 34 operated the real appliance, both comparing standard with enriched labelling. Enriched labelling helped in both, but paper prototypes may overestimate a design change's behavioural effect.
• Sauer, J. and Sonderegger, A. (2009). The influence of prototype fidelity and aesthetics of design in usability tests: effects on user behaviour, subjective evaluation and emotion. Applied Ergonomics. doi:10.1016/j.apergo.2008.06.006 Sauer and Sonderegger gave 60 participants a mobile phone at three levels of fidelity and two of visual appeal. Completion time may be overestimated with a computer prototype, and users appeared to compensate for deficiencies in aesthetic design by overrating the aesthetic qualities of reduced fidelity prototypes.
• Sauer, J., Sonderegger, A. and Schmutz, S. (2020). Usability, user experience and accessibility: towards an integrative model. Ergonomics. doi:10.1080/00140139.2020.1774080 Sauer, Sonderegger and Schmutz analyse the definitions and outcome measures of usability, user experience and accessibility, and argue for integrating the three as interaction experience, reported through spider charts. They supply the unit's account of what a prototype evaluation can claim.
• Sefelin, R., Tscheligi, M. and Giller, V. (2003). Paper prototyping - what is it good for? A comparison of paper- and computer-based low-fidelity prototyping. CHI Extended Abstracts on Human Factors in Computing Systems. doi:10.1145/765891.765986 Sefelin, Tscheligi and Giller ran test sessions with paper-based and computer-based low-fidelity prototypes, asking whether the medium changed participants' willingness to criticise the design. The medium was largely unrelated to it, and the accounts that cite the study report a user preference for the computer-based version.
• Virzi, R. A., Sokolov, J. L. and Karis, D. (1996). Usability problem identification using both low- and high-fidelity prototypes. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI). doi:10.1145/238386.238516 Virzi, Sokolov and Karis ran two experiments with two products, an electronic book and a voice response system, comparing problems found with low- and high-fidelity prototypes. Both found the same sets, with a significant correlation in detection proportions.
• Walker, M., Takayama, L. and Landay, J. A. (2002). High-fidelity or low-fidelity, paper or computer? Choosing attributes when testing web prototypes. Proceedings of the Human Factors and Ergonomics Society Annual Meeting. doi:10.1177/154193120204600513 Walker, Takayama and Landay tested sketched and HTML website prototypes in both paper and computer media, separating medium from fidelity. Low and high fidelity were equally good at uncovering usability issues, and results were independent of medium.