Tangible interaction is a thirty-year hypothesis with a thin evidence base

George Fitzmaurice, Hiroshi Ishii and William Buxton (1995) proposed that the hand should hold the interface, and Ishii and Brygg Ullmer (1997) argued that coupling digital information to physical objects would recover something the screen had thrown away. Thirty years on, the claim has a designer's grip on it, because an object feels as though it must beat a picture of one. This ninth of ten units argues that the founding claim has been stated far more often than it has been tested: where tangibility was compared against an equivalent graphical interface, the reliable difference lay in preference and engagement rather than in performance or learning. The claim has therefore shifted quietly from what the body can do to how the interaction feels.

The claim was made before it became a research programme

Fitzmaurice, Ishii and Buxton (1995) introduced graspable user interfaces: physical handles, which they called bricks, attached to virtual objects and moved by hand on a horizontal display surface. Their contribution was a concept and a design space, illustrated by exploratory studies of grasping, mock-ups and one working application. The argument is that physical handles carry affordances built from grip and reach that a cursor does not. Nothing in the paper tests whether the bricks beat a mouse.

Ishii and Ullmer (1997) generalised it. They argued that the graphical interface had discarded something valuable by confining interaction to a screen. They proposed coupling digital information to graspable objects, turning architectural surfaces into interfaces, and using ambient light and sound for background awareness. The position is illustrated by prototypes rather than by comparisons.

The field built frameworks before it built evidence

A decade of work went into describing the design space. Ullmer and Ishii (2000) proposed an interaction model relating physical and digital representations to physical control and the underlying model. Fishkin (2004) built a taxonomy on two axes, metaphor and embodiment, treating tangibility as a spectrum rather than a binary, so the usual comparison between a tangible interface and a graphical one is the wrong question; a system is more or less tangible, and the useful question is which degree and which property. Zuckerman, Arida and Resnick (2005) added an education-specific framework, separating manipulatives that model real structures from those that model abstract ones. Hornecker and Buur (2006) argued that earlier accounts under-described physical space and social interaction. These frameworks tell a designer how to compare systems; they do not say whether a system works.

The systems the frameworks describe are concrete enough. Underkoffler and Ishii (1999) built Urp, a planning workbench on which physical architectural models rest on an ordinary table. Projection above them throws shadows computed for any chosen time of day, reflects sunlight off models marked as glass, and drives a pedestrian-level wind simulation that stays aligned with the table. The design is a clear case of what the authors call luminous-tangible interaction, meaning distributed between physical objects and the light projected onto them.

The reviews asked the question, and the comparisons found effects other than comprehension

Marshall (2007) asked directly whether tangible interfaces enhance learning. He found that the conceptual work had mapped technical possibilities, while offering little guidance on the cognitive or social effects of choosing one interface over another. Shaer and Hornecker (2010), reviewing two decades, reached the same conclusion from a larger base. The field had produced many systems and comparatively few controlled studies of whether tangibility itself matters.

Horn and colleagues (2009) ran the comparison in a science museum. They observed 260 visitors and interviewed 13 family groups as they used tangible and graphical versions of the same programming exhibit. Visitors found the two equally easy to understand. With the tangible version they were significantly more likely to try the exhibit and to take part in groups, and children were more actively involved, an effect strongest for girls. The measured difference is in participation and engagement, not in comprehension. That difference comes from a self-selected sample, since a family that chooses to spend an hour in a museum can read the labels and reach the table.

Manches, O'Malley and Benford (2010) come closest to finding a performance effect. Three studies had children aged four to eight solve number-partitioning problems with physical blocks or on-screen equivalents. The physical materials did change the strategies the children used, but the active property was whether the pieces were moved individually or as a group. When that manipulation was constrained on screen, the gap narrowed. The advantage belonged to the properties of the representation, not to physicality as such.

Preference and performance are separate outcomes

Zuckerman and Gal-Oz (2013) compared tangible and graphical versions of the same modelling and simulation system, FlowBlocks. Most users preferred the tangible version, which keeps the founding claim alive and also narrows it. The two versions were equivalent on most performance parameters, and the tangible version was perceived as inferior on usability. Preference came from physical interaction, rich feedback and a sense of realism, not from getting the task done better or faster.

Hornecker (2011) gives the sharpest reading of this result. Reviewing what physicality actually contributes, she separates the properties that are genuinely physical from those that a screen could reproduce. She argues that much of the claimed benefit follows from the social and spatial embedding of the interaction rather than from the material. On this account the tangible advantage is real but mislabelled.

The contested question: real, absent, or mis-specified?

Three positions hold the field apart. On the first, the advantage is real and has been measured. Manches and colleagues (2010) found that physical representations changed children's problem-solving strategies. The frameworks of Zuckerman and colleagues (2005) and Ishii and Ullmer (1997) predict which tasks should benefit. On the second, the advantage disappears when the comparison is fair. Horn and colleagues (2009) found no comprehension benefit. Zuckerman and Gal-Oz (2013) found no performance benefit, and both Marshall (2007) and Shaer and Hornecker (2010) report that the field has rarely run the test. On the third, the dispute is mis-specified. Fishkin (2004) makes tangibility a spectrum and Hornecker (2011) locates the effect in social and spatial embedding, so a two-condition comparison measures the wrong variable.

What would settle it is a study that varies the degree of tangibility along Fishkin's axes while holding the task fixed. It would include an equivalent graphical condition, and report performance and preference separately. It would also have to control the social setting, since Hornecker and Buur (2006) show that shared physical space changes behaviour. Horn and colleagues (2009) come closest, but a museum is a place visitors are already disposed to enjoy.

The unit's own title carries a counter-argument. The evidence is thin for learning and performance, but not for engagement: Horn and colleagues (2009) observed 260 visitors and found the tangible version significantly more likely to be tried and to draw group participation, and Manches and colleagues (2010) measured a change in the strategies children used. The thinness belongs to the performance outcome rather than to the hypothesis, and the three positions are unevenly supported, since the first two rest on a handful of controlled studies and the third on frameworks rather than measurements. What a designer should do while the dispute is unresolved: name the property the physical form is meant to move, build the equivalent graphical condition, and report preference and performance separately.

Where this is heading

The direction of travel is towards specifying the material rather than asserting the benefit. Hornecker (2011) asks which physical properties do work and which are decoration. Sundstrom and colleagues (2011) propose building small demonstrators whose purpose is to expose what a technology can do, so that designers reason about properties instead of black boxes. Hoffard, Miyafuji and Koike (2025) present StringWall, a low-cost projection surface built from rubber strings, and report its design and interaction rather than its effect.

Sources

• Fitzmaurice, G. W., Ishii, H. and Buxton, W. A. S. (1995). Bricks: laying the foundations for graspable user interfaces. CHI '95. doi:10.1145/223904.223964 Introduced graspable user interfaces, in which physical handles called bricks are attached to virtual objects and moved by hand on a horizontal display. The paper sets out a design space and illustrates it with exploratory studies and one application rather than a comparison, so it is the founding statement of the claim this unit treats as untested.
• Ishii, H. and Ullmer, B. (1997). Tangible bits. CHI '97. doi:10.1145/258549.258715 Argued that the screen had discarded the physicality of interaction and proposed coupling digital information to graspable objects, architectural surfaces and ambient media. The position is illustrated with prototypes, including a table that senses physical objects, and not with measured benefit.
• Underkoffler, J. and Ishii, H. (1999). Urp: a luminous tangible workbench for urban planning and design. CHI '99. doi:10.1145/302979.303114 Built a planning workbench on which physical architectural models rest on an ordinary table while projected light throws shadows for any chosen time of day, reflects sunlight off glass facades, and drives a pedestrian-level wind simulation aligned with the table. The paper reports the interaction it affords rather than any measured benefit to planning.
• Ullmer, B. and Ishii, H. (2000). Emerging frameworks for tangible user interfaces. IBM Systems Journal. doi:10.1147/sj.393.0915 Proposed the MCRpd interaction model, relating physical and digital representations to physical control and the underlying digital model. The model organises existing systems and exposes which combinations had not been tried. It is a comparison tool and it does not test whether tangibility helps.
• Fishkin, K. P. (2004). A taxonomy for and analysis of tangible interfaces. Personal and Ubiquitous Computing. doi:10.1007/s00779-004-0297-4 Built a taxonomy whose axes are metaphor and embodiment, treating tangibility as a spectrum rather than a binary, the further from the origin the more tangible the system. The paper argues the spectrum unifies earlier definitions and suggests design principles for different regions. It is the reason a two-condition tangible-versus-graphical comparison is the wrong test.
• Zuckerman, O., Arida, S. and Resnick, M. (2005). Extending tangible interfaces for education: digital Montessori-inspired manipulatives. CHI '05. doi:10.1145/1054972.1055093 Proposed a classification of manipulatives into Froebel-inspired and Montessori-inspired kinds, and argued that the category a tangible interface belongs to determines what learning it can support. The framework is applied to computational building blocks rather than tested against a control. It shows the education claim is conditional on the kind of representation.
• Hornecker, E. and Buur, J. (2006). Getting a grip on tangible interaction: a framework on physical space and social interaction. CHI '06. doi:10.1145/1124772.1124838 Proposed a framework organised around physical space and social interaction, arguing that earlier accounts under-described both. The authors analyse case studies with the framework rather than running comparisons. It supplies the social setting a controlled comparison of tangibility has to control.
• Marshall, P. (2007). Do tangible interfaces enhance learning? TEI '07. doi:10.1145/1226969.1227004 Asked whether tangible interfaces enhance learning and reviewed the frameworks and the available evidence. The paper reports that the conceptual work had mapped technical possibilities while giving little guidance on cognitive or social effects, and it calls for more empirically grounded research. It is the field's own statement that the founding claim outran the evidence.
• Horn, M. S., Solovey, E. T., Crouser, R. J. and Jacob, R. J. (2009). Comparing the use of tangible and graphical programming languages for informal science education. CHI '09. doi:10.1145/1518701.1518851 Compared tangible and graphical versions of the same museum exhibit, observing 260 visitors and interviewing 13 family groups. The two were equally easy to understand, but the tangible version was significantly more likely to be tried and to draw group participation. The measured advantage is engagement, not comprehension.
• Shaer, O. and Hornecker, E. (2010). Tangible user interfaces: past, present, and future directions. Foundations and Trends in Human-Computer Interaction. doi:10.1561/1100000026 Reviewed two decades of tangible interface research, covering the interaction model, design space, enabling technologies and evaluation. The authors report that the field has produced many systems but comparatively few controlled studies of whether tangibility itself matters. It is the canonical statement of the descriptive bias.
• Manches, A., O'Malley, C. and Benford, S. (2010). The role of physical representations in solving number problems. Computers and Education. doi:10.1016/j.compedu.2009.09.023 Ran three studies in which children aged four to eight solved number-partitioning problems with physical blocks or on-screen equivalents. Physical materials changed the strategies the children used, but the active property was whether pieces moved individually or as a group. The physical advantage is real and traceable to the representation, not to physicality itself.
• Hornecker, E. (2011). The role of physicality in tangible and embodied interactions. Interactions. doi:10.1145/1925820.1925826 Reviewed what the physicality of tangible and embodied interaction contributes, separating properties that are genuinely physical from those a screen could reproduce. The paper argues that many claimed benefits follow from the social and spatial embedding of the interaction rather than from the material. It relabels the tangible advantage rather than denying it.
• Sundstrom, P., Taylor, A., Grufberg, K., Wirstrom, N., Solsona Belenguer, J. and Lunden, M. (2011). Inspirational bits: towards a shared understanding of the digital material. CHI '11. doi:10.1145/1978942.1979170 Argued that HCI often black-boxes the properties of a technology and proposed inspirational bits, small working demonstrators built to expose one or more properties of a digital material. The paper reports how the bits were used in design sessions to share design knowledge. It answers a field that asserts what a material does rather than specifying it.
• Zuckerman, O. and Gal-Oz, A. (2013). To TUI or not to TUI: evaluating performance and preference in tangible vs. graphical user interfaces. International Journal of Human-Computer Studies. doi:10.1016/j.ijhcs.2013.04.003 Compared tangible and graphical versions of the same modelling and simulation system, FlowBlocks, measuring performance, usability and preference. Most users preferred the tangible version, while the two were equivalent on most performance parameters and the tangible version was perceived as inferior on usability. Preference came from physical interaction, rich feedback and realism, and it did not track performance.
• Hoffard, J., Miyafuji, S. and Koike, H. (2025). StringWall: a low-budget, projection-based tangible user interface. Proceedings of the ACM on Human-Computer Interaction. doi:10.1145/3773072 Presents StringWall, a projection-based tangible surface made from low-cost rubber strings tracked by a depth camera and adaptable to the space it occupies. The paper reports the design, visibility tests and two preliminary studies of interaction preference. It continues the field's descriptive line rather than adding evidence about benefit.