Foundations of human-computer interaction

Human-computer interaction begins with an awkward fact. People do not use systems the way the systems were designed to be used. They skip steps. They invent workarounds. They misread the label, then blame themselves. Those behaviours are predictable to a degree. Many are designed in.

The field studies the fit between people and their machines, and that fit outlives the technology. Human capacities change slowly, though they vary between people and across a lifetime. Attention is limited. Working memory holds about four items at once on average. One review collects estimates of three to five, and finds the limit only under stated conditions (Cowan, 2001). Those constraints are the stable part, and what the field disputes is how to explain them.

The joint system is the unit of analysis

The first move in HCI is to refuse to study the computer alone or the person alone. The unit is the joint system: a person pursuing a goal with a machine in a setting. Change any one, and the interaction changes.

In the 1960s the United States Air Force built cockpit warning systems. A recorded voice told the pilot to turn, climb or dive to avoid a collision (Harrison, Tatar and Sengers, 2007). The Air Force picked a woman's voice so it stood out as the emergency voice. It worked while every pilot was a man. Then women became pilots and flight controllers, and the same recording changed meaning. The software had not changed. The people in the cockpit had.

A laboratory test speaks about the conditions it held, and a specification names only the users its authors imagined.

Each origin story left an assumption behind

The field reached the joint system in stages, and each stage left an assumption behind (Myers, 1998).

The first idea is linking. Vannevar Bush's memex essay of 1945 is credited with the idea of linked documents, and Engelbart's NLS system used links from 1965. Tim Berners-Lee built the web on the idea at CERN in 1990 (Myers, 1998).

Pointing and windowing came next. Sutherland's Sketchpad was his 1963 doctoral work at MIT, and a light pen could grab, move and resize objects (Sutherland, 1963). The mouse was built at Stanford Research Laboratory in 1965, and Engelbart showed it with tiled windows in 1968. The Xerox Star used the style commercially in 1981, and the Macintosh followed in 1984 (Myers, 1998).

The latest turn is generative. A person can now state an outcome in ordinary language and the machine produces it, rather than naming every step. The guidance for such systems is a proposal, not a measured effect (Weisz et al., 2024).

One assumption runs through them: a user alone with a screen and a fast connection.

Two gulfs locate failure, and the account behind them is disputed

People outlive the tool, and Norman's account of action is the instrument the field reaches for first (Norman, 2013). A person forms a mental model of how a system works. That model is incomplete, often wrong in detail, and still good enough to act on. Interaction fails at two points.

The gulf of execution is the distance between what a person intends and what the interface lets them express. You want the font smaller. Is there a control? Is it where you look? Is its name recognisable, and in a language the person reads?

The gulf of evaluation is the distance between what the system did and what the person can tell it did. You pressed the button. Did it save, queue, or fail? That produces uncertainty after acting, and people press buttons twice.

Xerox PARC built Bravo in 1974, the first editor to show the page as it prints (Myers, 1998). Before Bravo, users got what they had described, not what they had seen, so a visible page shortens the gulf of evaluation.

That account of action is contested. Suchman argues that action is organised in the setting. A plan, she writes, is a conceptual and rhetorical device, and people organise action in its terms (Suchman, 1993). She answers Vera and Simon, who held that situated claims could be met inside a symbolic framework (Suchman, 1993).

The two sides disagree about evidence too. A cognitive account predicts performance in a controlled task, and a test can show it wrong. A situated account shows what people did in one setting and predicts nothing for an unbuilt screen.

Treat a mental model as a hypothesis about your users, not a description of the mechanism, and test it where the work happens.

An affordance is a possibility, not a cue

Gibson introduced the term affordance (Gibson, 1979). An affordance is a property of an environment that offers an animal an action, relative to that animal's capabilities. A flat, rigid surface affords standing to a creature with legs, whether or not anyone perceives it.

Norman adapted the term for design and later regretted it. Designers had used it for the visual cue rather than the underlying possibility, so he proposed signifier for the cue. The underline signals a link, the shadow a raised button, the handle a draggable edge (Norman, 2013).

Signifiers can be borrowed. The Macintosh pushed its toolkit on other developers to enforce one interface (Myers, 1998). What a user learns in one place carries into the next.

A signifier works only if a person perceives it, and perception varies between people and within one person. A screen reader user, a person in bright sun or without their glasses can miss it. Remove the outline from a control, and the pressable-looking cue goes with it. That is signifier loss, and use is expected to fall.

That expectation is a prediction, not a result, and nothing in these sources measures the fall. One counter deserves an answer. If a cue changes no behaviour for the people observed, it carried nothing for them, so removing it costs nothing. That counter is an argument, and so is the prediction. A comparison of one task with and without the cue would settle the question if it counted the errors and named who was counted.

Direct manipulation, and the costs it lowers

Shneiderman's 1983 paper named what graphical interfaces were already doing (Shneiderman, 1983). The system represents the object of interest continuously. The user acts through physical actions or labelled buttons rather than complex syntax, so a keyboard route sits inside the style. Every operation is rapid, incremental and reversible, and its effect is visible at once.

VisiCalc shows why. Bricklin and Frankston wrote it for the Apple II in the late 1970s. A cell held a value or a formula, so editing one changed every cell that depended on it. A user could ask what a price rise would do without programming.

Dragging a file into a folder is the same move. Typing mv report.txt archive/ is not. The paper lists problems as well as properties. Graphic representations take screen space. Experienced users may prefer a table of fifty document names to ten truncated icons (Shneiderman, 1983). The same paper claims experts on these systems work extremely rapidly. Both claims come from examples and from studies by others, so the expert case is argued in both directions and measured in neither.

The reversibility clause is most often dropped. A drag that can be undone is direct manipulation. A drag that deletes permanently when it lands in the wrong place only looks like direct manipulation and carries the risk of a command line. The argument is about the cost of learning and the cost of error, which fall hardest on occasional users.

Three paradigms for placing work, not for ranking it

Those disputes sit in different paradigms. Harrison, Tatar and Sengers described three paradigms in 2007. They draw the difference with the cockpit voice from the first section (Harrison, Tatar and Sengers, 2007).

Read as human factors, the problem is a critical incident. A woman's voice got the pilot's attention at once. The paradigm couples person and machine and judges a fix by whether it worked. Its weakness is repair without an account of why it worked.

Read as classical cognitivism, the voice is a channel and the question is signal against noise. The paper reports that familiar voices improved pilot performance in experiments. The paradigm models mind and computer as coupled information processors. GOMS (goals, operators, methods and selections) sits in that tradition (Harrison, Tatar and Sengers, 2007). Its weakness is what it marks as marginal: emotion, meaning and the detail of a setting count as noise unless they move the measurement.

Read as phenomenologically situated, the voice was reputed to have been chosen for a seductive tone. That choice made the cockpit a male space, and it became inappropriate once women flew and controlled (Harrison, Tatar and Sengers, 2007). Here interaction is meaning making and the setting carries part of the explanation. The paper's reference for that is Suchman's study of plans and situated actions (Suchman, 1993).

The cockpit case is the paper's own cartoon rather than a study, and its claim about familiar voices carries no source. The paper does not rank them, and says the three can coexist. The useful question is which one a piece of work belongs to.

Usability and user experience are not synonyms

The standards fix terms the paradigms leave open. ISO 9241-11 gives the field its definition of usability (International Organization for Standardization, 2018).

The extent to which a system, product or service can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use.

The standard adds that usability is an outcome of use, in a context that covers the device, the connection and the user's ability. Four phrases carry the weight: specified users, specified goals, effectiveness and efficiency, and a context of use. Remove any one, and the claim becomes untestable.

The standard states its boundary: it defines the concept and describes no methods (International Organization for Standardization, 2018). A usability claim therefore borrows a method, and that choice is where argument starts.

PROMIS was the first hypertext system released to its user community in 1976. It linked patient and care information at the University of Vermont's medical centre (Myers, 1998). Against the definition, the users are clinicians and the goal is to reach a record. Myers records no usability data, so the case shows a testable claim, not a result.

ISO 9241-210 is the human-centred design standard. It defines user experience as a person's perceptions and responses. These result from the use or anticipated use of a system, product or service (International Organization for Standardization, 2019). Emotion, belief, preference and comfort count, and so does anticipation before use. Experience is broader than usability and harder to measure. That phrase is an inference here, not a quotation. The standard lists what counts as a response, and does not rank them by how easily they can be observed.

A claim about experience needs a method that can support it, and that choice is argued about rather than settled.

Where failure can injure, usability is risk management

The stakes are unevenly distributed, and that unevenness is the ethical core of the discipline. A poorly designed benefits claim form can exclude someone from money they are entitled to. The person filling it in can read slowly, read in a second language, or share one metered phone. Those are ordinary conditions of use. Where a failure can injure a person, the cost is not measured in inconvenience.

IEC 62366-1 shows how the field treats that case (International Electrotechnical Commission, 2020). It specifies a process by which a manufacturer analyses, specifies, develops and evaluates device usability as it relates to safety. The manufacturer also assesses and mitigates the risks of correct use and of use error. Usability there is risk management, not a satisfaction score. The standard states its own boundary: it addresses normal use and cannot assess or mitigate the risks of abnormal use.

A designer who can only produce a preference is not equipped for the work.

What the rest of the programme assumes from here

You now have the vocabulary and the diagnostic instruments. The next module gives you the ability to build, and a designer who cannot implement has to negotiate rather than decide. Module five, on perception and cognition, returns to this material from the psychological side, and asks how strong the evidence behind each design rule is. The disputes named above are the first entries on that list.