Theories of interaction

A theory of interaction is a claim about where the explanation of an interaction lives. Put it in the person's procedure and you get a predictive model. Put it in the arrangement of people, screens and paper, or in the wider activity, or in the occasion, and you get another account of the same five minutes. The choice decides what you notice and what you treat as noise.

The contested question runs through the area. Must a theory predict something measurable, or is it enough that it makes a setting intelligible and changes what a designer does next? The predictive tradition held the field's centre for two decades and has a track record the interpretive traditions lack, while those traditions answer questions it cannot pose. Rogers (2012) charts what happened as the field widened: theory came to be used for describing, explaining and critiquing, and for generating designs, as well as for testing hypotheses about performance. The dispute is live, and better taught as a dispute than as a succession of schools.

Every theory below is applied to a case and judged by what it revealed, and by what it cannot claim.

GOMS predicts a time you can check, and that is why it survives

The information-processing tradition treats person and machine as coupled processors and seeks predictions before anything is built. Card, Moran and Newell gave that programme its most durable instrument. A GOMS model describes skilled use in four parts: goals, operators, methods, which are sequences of operators that accomplish a goal, and selection rules that choose between methods (John and Kieras, 1996).

John and Kieras analyse one task, moving a marked-up phrase in a word processor, in four variants. The simplest, the keystroke-level model, totals pre-set times for an analyst-supplied operator sequence and predicts 14.38 seconds. The original formulation predicts the same. A structured language variant, NGOMSL, predicts 16.38 seconds and also 784 seconds of procedure learning. A parallel-activity variant puts the highlighting sub-method at 2.21 seconds (John and Kieras, 1996).

The numbers rest on an assumption about the person. The rationality principle says users settle on methods that are efficient given the interface, so a careful description of the interface predicts the procedure (John and Kieras, 1996). Where a task is repeated, practised and uninterrupted, the model earns its place: it compares two designs before either exists.

The authors state the limits. The simple variants assume serial processing, with no parallel activity, no interruption and no interleaving of goals, and the validated predictions sit at the keystroke level (John and Kieras, 1996). The model says nothing about error, discovery or emotion, and cannot tell you whether the method the analyst supplied is the one anybody would use.

Distributed cognition moves the unit of analysis off the skull

Cognitive science normally takes the individual as its unit of analysis. Hutchins showed that the cognitive properties of a whole system can differ from those of the people inside it, using a memory task in the cockpit of a commercial airliner (Hutchins, 1995).

The case is the airspeed and wing-configuration speeds a crew must compute and hold through an approach. The speed card booklet is a long-term memory for the cockpit system, and the speed bugs, the markers that move round the dial, are not aids to individual remembering. They are part of the process by which the cockpit system remembers: the speeds are read, written and compared across many representations (Hutchins, 1995).

The design consequence is testable. Move the bug setting into a menu and you have changed the memory system, not only the display: the record of the speeds was carried by the arrangement of the workspace, so that arrangement belongs in the interface specification. The limit mirrors the strength: the claim generalises by analogy from one detailed study, and the same label names both a theory of cognition and a method of description.

Activity theory asks what the activity is for, and where it strains

Activity theory starts from Vygotsky's argument that action on an object is mediated by cultural artefacts, so the unit of analysis is subject, object and artefact together rather than the isolated mind. Leont'ev added the distinction between an individual action and the collective activity that gives it point (Engestrom, 2001).

Engestrom (2001) summarises the tradition in five principles: the activity system is the unit of analysis, activity is multi-voiced, it has a history that explains its troubles, contradictions drive change, and change can take the form of an expansive cycle. In its third generation the minimum unit is two interacting activity systems.

His case is an intervention study in multi-organisational medical care for children in the Helsinki area. The object of the work moves from an unreflected case, to a patient treated as an instance of a disease category, to a jointly constructed understanding of the child's situation and care plan (Engestrom, 2001). Disagreements between clinics then stop looking like communication failure and start looking like two systems with different objects.

Kaptelinin and Nardi (2006) make the tradition usable. They set out its principles and history, show the object of activity used in an empirical study, and compare it with its competitors in interaction design, distributed cognition and actor-network theory. Their questions for a designer are what the object of this activity is, whose it is, which artefacts and rules mediate it, and who the community is. The framework interprets rather than predicts.

Situated action says the plan is the account, not the mechanism

Suchman's target was the model that treats a course of action as the execution of a plan, a model then being built into machines (Suchman, 1987). Her contrast case is navigation at sea: the European navigator begins with a charted course and relates every move to it. The Trukese navigator begins with an objective and steers by wind, waves, stars and the feel of the boat, and can point to the objective at any moment but cannot describe a course.

The claim is stronger than one about culture. All activity, even the most analytic, is fundamentally concrete and embodied, so purposeful actions are inevitably situated actions. Plans are best viewed as a weak resource for what is primarily ad hoc activity: stated in advance they are vague, and reconstructed in retrospect they filter out the particularity of what was done (Suchman, 1987).

The consequence for design is precise. A system that requires people to declare their plan before it will help them, and treats deviation as error, fails exactly on the occasions that matter. In a constructed case, a ticket kiosk opens with four choices of journey type when the traveller's problem is a name on the booking that does not match the passport. Nothing has been misunderstood; the design has nowhere to put "this is not my usual situation". The critique is contested, because a procedure model can still be validated on the routine cases it covers.

Embodiment puts meaning in the world rather than in the head

Dourish traces the philosophical foundations of interaction design to the phenomenological tradition, especially Heidegger, Schutz and Wittgenstein (Dourish, 2001). Its central move is that the philosophical puzzle case, the thinking problem-solver, is the exception. Ordinary activity is absorbed, skilled and unreflective, and that is what a theory of interaction must explain.

His proposal is embodied interaction, organised around the creation, manipulation and communication of meaning, and the establishment and maintenance of practice (Dourish, 2001). Tangible computing, which moves interaction off the screen into physical objects, and social computing, which treats social meaning as part of the design problem, share that foundation in his account. The claim about practice is where the social turn meets the interface: what people can do depends on a competence held with others and maintained in a setting.

A constructed case shows what the theory predicts. A studio mixing desk gives each channel a fader whose position is readable without looking and is visible to a colleague behind you. A touchscreen mixer puts the state behind a tap and demands visual attention for every adjustment, in a task that already uses the ears and the conversation. The theory predicts that the physical control will be kept where attention is needed elsewhere, and that a gesture with no physical anchor will go undiscovered. The book's design principles are argued and illustrated rather than tested, and so broad a term needs its sense spelled out in any claim that a design is embodied.

Socio-technical design refuses to separate the technology from the organisation

The socio-technical argument is that the technical system and the work organisation are one design problem, because changing the technical system changes the work, the roles and the handling of exceptions. Baxter and Sommerville (2011) analyse why the approach is widely acknowledged and not widely practised, and propose a framework joining organisational change and system development through sensitisation and awareness, and constructive engagement.

In a constructed case, an electronic prescribing system passes its acceptance tests: every function works and the documented tasks take less time. On the ward the pharmacy's checking step now happens in a different order, and the exception path for an unusual dose runs through nobody's screen. An exception that an experienced person used to absorb on the walk to pharmacy has become an outage.

The tradition supplies a way of scoping a project and a warning about what acceptance testing does not cover. The framework is argued from a review of design methods rather than validated as an intervention, which is a reason its authors give for its limited take-up.

Actor-network theory, used carefully, keeps the artefact in the account

Latour argues that sociologists kept failing to balance their accounts of society because the missing mass was in things. His object is an automated door-closer and his problem is a seat belt that will not let the car be driven unbuckled: engineers link the ignition to a sensor in the belt, and a rule about behaviour becomes a property of the assembly of driver and car (Latour, 1992).

What is useful for a designer is the vocabulary. An artefact can be made to carry an action on someone's behalf, and design can replace human work and constrain what other people do. A door-closer holds a door that would otherwise be held by a person; a default in a form is a decision made once and then enforced on every later user. The question this makes askable is what a component is doing, and for whom.

The caution is that the theory was built to describe science and technology in the making and offers no design criteria, while its symmetry between human and non-human agency can be used to justify almost anything. Kaptelinin and Nardi (2006) treat it, alongside distributed cognition, as a competitor that activity theory must answer.

The third wave, and what a theory has to do next

Bodker (2006) describes the second wave of HCI as work-centred: groups using a collection of applications in established settings, analysed with situated action, distributed cognition and activity theory, and designed through participatory workshops. In the third wave the use context and application types are broadened and intermixed, technology spreads from the workplace into homes and culture, and elements such as culture, emotion and experience enter the field. Her topics are multiplicity, context, boundaries, experience and participation.

Two observations are usually left out of the slogan version. Third-wave research is defined partly in the negative, as non-work, non-purposeful and non-rational. And it has partly moved from a commitment to users towards a more exploratory stance, taking inspiration from use rather than designing with users.

Has usability been superseded? The case for it is that effectiveness and efficiency for specified users in a specified context say nothing about meaning, emotion or culture, and that conditions of use no longer resemble a workstation and a task. The case against is that a broader question does not answer the narrower one, and that an experience claim still needs a method able to support it. The defensible position is that usability was outgrown as a sufficient account rather than refuted.

Rogers (2012) surveys the same terrain as classical, modern and contemporary theory. The answer implied by these cases is that a theory keeps its role while it changes what can be said about a case. The next problem goes beyond the theories gathered here: a system that learns does not fix its output for a given input, and a theory of that interaction has to explain a moving target.