What the paper claims: contribution types and the terms of judgement

A project is judged twice. A reader asks whether it was done well, and whether what it claims to have produced is worth having. The second judgement follows a standard that the work itself selects.

The first of ten units argues that a reader judges a paper against the kind of knowledge it claims to contribute, and it sets the terms the later units inherit. A project that names its type in advance inherits the criteria it will be held to, and it can then use those criteria to bound what it asserts. The type is a plan made before the study, not a label attached at the end.

The reviewer's first question is what kind of knowledge this is

Wobbrock and Kientz (2016) set out the account of research contribution that HCI now uses. They distinguish seven types of knowledge, empirical, artefact, methodological, theoretical, dataset, survey and opinion, and each carries characteristics that imply how it is judged. Reviewers judge empirical contributions mainly on the importance of their findings and the soundness of their methods. They judge methodological contributions on the utility, reproducibility, reliability and validity of the method, which requires repeated validation. They judge theoretical contributions on novelty, soundness and their power to describe, predict and explain, and they demand that a theory be testable and falsifiable. They judge artefact contributions according to what kind of artefact gave rise to them (Wobbrock and Kientz, 2016).

Each contribution type has key characteristics that imply how it is judged (Wobbrock and Kientz, 2016).

The field has adopted the taxonomy as its own instrument. The CHI call for papers tells authors that a paper must make an original research contribution. It directs them to Wobbrock and Kientz for the types, and it lists the types it recognises with the questions a reviewer may use to assess each (ACM CHI, 2027). Two of those questions belong at the planning stage. Do you clearly review what is already known and what limits exist in that knowledge? Do you describe what you made in sufficient detail for others to replicate it (ACM CHI, 2027)?

The field's own numbers show what it rewards, not what it values

Wobbrock and Kientz set out seven types of knowledge. The CHI 2016 submission form offered eight contribution keywords, because it divided empirical work into studies of people and studies of system use. Across those eight keywords, empirical studies of people had the best acceptance rate of any type, at 26.3 per cent. Dataset contributions were the least submitted, at 1.3 per cent, and also the least accepted, at 16.7 per cent (Wobbrock and Kientz, 2016).

Those figures tempt a candidate to pick the type most likely to be accepted, and the authors warn against that reading. The counts describe the reviewers of one English-language conference, not the value a field places on a contribution type once it is published. Method and theory papers may be hard-won and relied on for years (Wobbrock and Kientz, 2016). Name the type you are claiming, then pay what that type owes. An artefact claim owes a description detailed enough to replicate. It also owes a validation appropriate to the kind of artefact it is (ACM CHI, 2027).

Interpretive contributions need interpretive criteria

A cluster of work shows what happens when a contribution is judged by a standard written for another type. Gaver, Beaver and Benford (2003) argue that ambiguity, usually treated as a defect in HCI, can be designed in deliberately as a resource for close personal engagement. They distinguish three classes of ambiguity according to where the uncertainty sits between person and artefact. Sengers and Gaver (2006) extend the argument to evaluation. A system designed to support multiple competing interpretations is misjudged when it is assessed against a single authoritative meaning. Bertelsen and Pold (2004) propose interface criticism as a way of analysing an interface as a cultural artefact, and they argue that mainstream approaches lack the openness to assess experience. Bardzell and Bardzell (2015) give the general case. They contrast the scientific report with the humanistic essay as research products, and they argue that each needs its own account of rigour.

Gaver, Beaver and Benford (2003) give the type a design to inspect. Their Home Health Monitor reads light, temperature and door sensors in a house, and it maps the readings onto sentences drawn from published horoscopes to produce a daily forecast for the household. The mapping is deliberately imprecise, so the reader has to decide how much credence to place on the forecast, a judgement that assumes the household can read it in its own language. The design fails judged by whether users converged on one reading, and it succeeds judged as a resource for interpretation.

The consequence is concrete. If the contribution is interpretive, the evidence is the argument and the reading it supports, not a measured effect. The paper must say so rather than borrow the empirical form. Judging an ambiguity-led design by whether users converged on one interpretation is a category error, not a weak result.

A theory reorganises a concept rather than predicting an outcome

Rogers (2012) charts how the role of theory in HCI expanded from the early days of scientific testing to include describing, explaining, critiquing and generating designs. Hornbaek and Oulasvirta (2017) show what that breadth costs. Interaction is the field's defining term, yet their review finds few direct attempts to define it and several incompatible concepts of what it is. Each concept has a different scope and a different assumption about causality between human and computer, which limits its ability to inform studies and designs. They catalogue them as interaction seen as dialogue, as transmission of information, as tool use, as optimal behaviour, as embodiment and as control. Interaction as dialogue treats the exchange as turns with mapping and feedback, and it rates a system by how understandable it is. A theory contribution, on this showing, earns its place by making a core concept usable across many studies, not by predicting one measured outcome.

The contested question: is the type a label or a plan?

The literature agrees that contribution types matter and divides over whether the list is complete. Wobbrock and Kientz (2016) and the CHI criteria (ACM CHI, 2027) hold that the types are the field's own account of the knowledge it produces, so naming one inherits its criteria. A second position holds that forcing a study into a type can misdescribe it. Dourish (2006) argues that ethnography contributes more than a set of requirements. Fallman (2003) separates the knowledge-generating conduct of design-oriented research from the artefact-generating conduct of research-oriented design. Nelson and Stolterman (2012) give the wider version. Design is its own culture of inquiry, with its own epistemology, and it is not a branch of science judged by scientific standards. Reeves (2015) reaches a similar worry from inside the field's history, asking how HCI's notions of rigour and disciplinarity should now be reassessed.

A third position holds that the list is incomplete at the middle. Hook and Lowgren (2012) propose strong concepts, forms of knowledge more abstract than a particular instance but not aspiring to the scope of a general theory, illustrated by social navigation and seamfulness. Such knowledge is generative and cuts across use situations, and it concerns interactive behaviour rather than static appearance, yet none of the seven types names it. Stolterman and Wiberg (2010) arrive at the same place from the design side. They describe a concept-driven approach in which theoretical constructs appear in concrete designs.

No study settles the dispute. A study of reviewer behaviour would settle it. It would take papers that declare a type, measure whether reviewers apply the questions the type attracts, and test whether the declared type predicts the outcome.

Read by kind, the two sides carry different evidence. The first rests on a taxonomy distilled from published contributions and on a conference page that states a policy. The second rests on arguments from ethnography and from design practice. Neither is a study of what reviewers do. While the dispute is open, a project should name its type, state the criteria it accepts, and say how it would sit if the type were read as a label. The strongest objection to this unit's thesis is that an author may name the type at the end and still be judged on the work, so early naming governs the author rather than the reviewer.

Where this is heading

The taxonomy is expanding rather than settling. The current CHI contribution page adds a type the 2016 account did not list, validation and replication. It asks authors to say what is the same and what has changed in the method since the original study (ACM CHI, 2027). Hook and Lowgren (2012) press for intermediate-level design knowledge to be treated as a knowledge form in its own right. Hornbaek and Oulasvirta (2017) press for a shared theory of interaction. Both moves point the same way: the field keeps adding kinds of knowledge it wants judged on their own terms. Whether more types sharpen judgement or dilute it is the open question a project inherits when it chooses the type by which it will be judged.

The sources this unit rests on

Each note states what the cited work did and found and what it adds to the argument. Titles, years and identifiers were checked against registered metadata on 31 October 2026.

• Wobbrock, J. O. and Kientz, J. A. (2016). Research contributions in human-computer interaction. Interactions, 23(3), 38-44. doi:10.1145/2907069 Sets out seven contribution types and the criteria each attracts, then reports the CHI 2016 distribution: 44.0 per cent of 2,316 submissions were studies of system use and 28.4 per cent were studies of people, whose 26.3 per cent acceptance rate was the highest of any type against a conference rate of 23.6 per cent, while dataset contributions were least submitted (1.3 per cent) and least accepted (16.7 per cent). It is the unit's account of what a type is for: naming one inherits the questions by which it will be judged.
• ACM CHI (2027). Contributions to CHI. Conference page for the ACM Conference on Human Factors in Computing Systems. https://chi2027.acm.org/contributions-to-chi/ States the contribution types CHI recognises, the reviewer questions attached to each, and directs authors to Wobbrock and Kientz for the types. It carries the unit's claim that the type is the reviewer's instrument, not only the author's label.
• Hook, K. and Lowgren, J. (2012). Strong concepts: intermediate-level knowledge in interaction design research. ACM Transactions on Computer-Human Interaction, 19(3), 1-18. doi:10.1145/2362364.2362371 Proposes strong concepts as knowledge above the particular instance but below general theory, illustrated with social navigation and seamfulness. It is the unit's case that one kind of design knowledge fits none of the seven types cleanly.
• Stolterman, E. and Wiberg, M. (2010). Concept-driven interaction design research. Human-Computer Interaction, 25(2), 95-118. doi:10.1080/07370020903586696 Describes a concept-driven approach that manifests theoretical constructs in concrete designs and integrates it with user-centred practice. It gives the unit a design-side route to the same intermediate knowledge.
• Dourish, P. (2006). Implications for design. ACM CHI '06, 541-550. doi:10.1145/1124772.1124855 Argues that treating implications for design as the main criterion for ethnographic work fails to capture what such investigations contribute. It is the unit's evidence that the criteria attached to a requested contribution type can misrepresent the study.
• Fallman, D. (2003). Design-oriented human-computer interaction. ACM CHI '03, 225-232. doi:10.1145/642611.642652 Sets out three accounts of what design is and distinguishes knowledge-generating design-oriented research from the artefact-generating conduct of research-oriented design. It gives the unit its example of two activities with two standards of judgement.
• Reeves, S. (2015). Human-computer interaction as science. Aarhus Series on Human Centred Computing, 1(1), 12. doi:10.7146/aahcc.v1i1.21296 Traces cognitive science's influence on HCI through early input-device design and the idea of a scientific design space, and asks how the field should reassess rigour and its disciplinarity. It carries the unit's challenge to the assumption that one scientific standard governs every contribution.
• Rogers, Y. (2012). HCI Theory: Classical, Modern, and Contemporary. Synthesis Lectures on Human-Centred Informatics, 5(2), 1-129. doi:10.2200/s00418ed1v01y201205hci014 Charts how the role of theory in HCI expanded from scientific testing to describing, explaining, critiquing and generating designs. It shows the unit that a theory claim and an empirical claim are judged by different tests.
• Gaver, W. W., Beaver, J. and Benford, S. (2003). Ambiguity as a resource for design. ACM CHI '03, 233-240. doi:10.1145/642611.642653 Argues that ambiguity can be designed in deliberately and distinguishes three classes of ambiguity by where uncertainty sits between person and artefact. It is the unit's case for a contribution whose value is interpretive rather than measured.
• Sengers, P. and Gaver, B. (2006). Staying open to interpretation: engaging multiple meanings in design and evaluation. ACM DIS '06, 99-108. doi:10.1145/1142405.1142422 Shows how design and evaluation strategies shift once the goal is multiple coexisting interpretations rather than one authoritative reading. It supplies the unit's argument that the criteria must follow from the kind of thing claimed.
• Bertelsen, O. W. and Pold, S. (2004). Criticism as an approach to interface aesthetics. ACM NordiCHI '04, 23-32. doi:10.1145/1028014.1028018 Proposes interface criticism as a way of analysing an interface as a cultural artefact and offers a criticism guide, arguing mainstream approaches cannot assess experience aspects. It is the unit's worked non-empirical contribution type.
• Bardzell, J. and Bardzell, S. (2015). Humanistic HCI. Synthesis Lectures on Human-Centred Informatics. doi:10.1007/978-3-031-02214-2 Reintroduces humanistic ways of knowing to HCI and contrasts the scientific report with the humanistic essay as research products, with advice for reviewing each. It carries the unit's case that products of different kinds need different accounts of rigour.
• Nelson, H. G. and Stolterman, E. (2012). The Design Way: Intentional Change in an Unpredictable World. MIT Press. doi:10.7551/mitpress/9188.001.0001 Argues that design is its own culture of inquiry and action, with a core of ideas distinct from the scientific tradition. It gives the unit the philosophical backing for judging design contributions on their own terms.
• Hornbaek, K. and Oulasvirta, A. (2017). What is interaction? ACM CHI '17, 5040-5052. doi:10.1145/3025453.3025765 Reviews how interaction has been conceived, finding few direct definitions and several incompatible concepts, and lists desiderata for future work. It gives the unit its example of a theoretical contribution that reorganises a core term.