What counts as a contribution: seven types, and the arguments about them

Every research paper claims knowledge, and the field keeps a short list of the forms that claim may take. The flagship conference has adopted the list, and readers misread it as a test. This unit argues that the seven contribution types are a working vocabulary rather than a settled standard. A community's judgement of value decides what counts as a contribution, and not the type a paper wears. A researcher must therefore argue for the kind of claim they make. The design literature holds that the list cannot contain what making produces. The practice literature holds that a second community has a veto. The paradigm argument holds that one paper can be knowledge in one tradition and not in another.

Seven types, and the machinery that adopted them

Wobbrock and Kientz (2016) set out the vocabulary. There are seven contribution types: empirical, artefact, methodological, theoretical, benchmark or dataset, survey, and opinion. Each type implies the test it faces. A finding faces its importance and the soundness of its method, an artefact what it makes possible, and a method its utility. A theory faces its explanatory power, a dataset its usefulness, a survey the organisation it achieves, and an argument its strength. The list did not stay a description. CHI 2016 made the declaration optional and offered up to eight checkboxes instead. Of the 2,316 submissions, 1,481 indicated exactly one contribution type, 705 indicated more than one, and 130 indicated none (Wobbrock and Kientz, 2016). There were eight rather than seven because the empirical type was split between studies of system use and of people. The vocabulary is an instrument edited to fit the work. It has since been redrawn: the CHI 2024 submission guide sets out eight contribution types of its own, among them understanding users, methodology, argument, and validation and replication (ACM SIGCHI, 2024). Each programme committee restates the list rather than inheriting it. The instrument also assumes a paper written in English, so a writer in a second language meets it on harder terms.

We stress that the level of research activity should not be confused with the value placed on particular contribution types once they are published. (Wobbrock and Kientz, 2016)

A theoretical contribution changes the object of study

Theoretical work resists the test most visibly. Hollan, Hutchins and Kirsh (2000) argue that the single-user, information-processing picture ran out of room as networked computers spread beyond the desktop. They propose distributed cognition as a new foundation. Its focus has always been on whole environments and how activity is coordinated within them. It reports no experiment and no effect size. Rogers (2004) adds the audience problem. She finds that the theories imported into HCI broadened the field's discourse while having limited impact on the practice of interaction design. A theoretical contribution therefore answers to more than one audience.

A contribution can be a thing, and what it makes is the test

Fallman (2003) states the design position early. HCI had become a design-oriented field whose philosophical and methodological basis had not kept pace. He separates knowledge-generating design-oriented research from artefact-generating research-oriented design. Zimmerman, Forlizzi and Evenson (2007) turn that into a method. In research through design, designers make novel integrations of prior research to make the right thing, and the making is the research. They formalise the model with four lenses and three examples, and argue that design earns its place on under-constrained problems. Hook and Lowgren (2012) name the level such work produces. A strong concept is more abstract than an instance and less general than a theory, generative, and carried by a core design idea that crosses use situations and domains. They present two, social navigation and seamfulness, and ask that such knowledge be contestable and substantive. Seamfulness is worked through at length. It began as a counter-reaction to the field's pursuit of hiding the seams in network coverage and positioning. The paper's instance is the signal-strength indicator on a mobile phone, which shows what the user would notice anyway and then becomes a tool for moving to better coverage. Hook and Lowgren find the same move in Facebook place labels, where users name a position. What the case does not show is a measured benefit, which the paper does not claim. Gaver (2012) sharpens the edge. He concludes that research through design yields provisional, contingent and aspirational theories, and proposes that theory be treated as annotation of portfolios. That is a direct objection to a taxonomy built to standardise the form a claim takes.

The second community: practice judges too

Sutcliffe (2000) shows that the test of a contribution can be another person's use of it. He finds direct application of cognitive theory to design limited by scalability problems. He proposes claims as a bridging representation that lets theory frame recommendations for designers, though claims are grounded in specific scenarios and so have limited generality. Gray, Stolterman and Siegel (2014) interviewed interaction designers about their use of design methods, including affinity diagramming and affordance. They built an account of a bubble-up of ideas from practice into research and a trickle-down of theory into practice. Value is therefore partly conferred by a community that does not review papers.

Contested question: is the test the type, the paradigm, or the community?

One position holds that the type is the test. Wobbrock and Kientz (2016) state that each type implies how it is judged, and CHI 2016 then implemented the types as a submission instrument. Reading a paper as examined against the standards of its declared type is the position's step, not the paper's claim.

A second holds that the paradigm is the test, the type downstream of it. Harrison, Tatar and Sengers (2007) argue that HCI contains three coexisting paradigms: human factors, classical cognitivism and a phenomenologically situated third. Each is a world view with its own criteria for what counts as knowledge. They run one cockpit example through all three, the warning systems built in the 1960s. Under the first paradigm, a recorded female voice reduced pilot errors. Under the second, familiarity of the voice was optimised against mental workload. Under the third, the meaning of a cockpit coded as male became the problem. Dourish (2006) adds the related point about a method, arguing that the demand for implications for design misreads what an ethnographic study is for. In both cases the community's demanded form of inference fixes the claim.

A third position holds that the test is neither, but uptake. Sutcliffe (2000) locates value in whether designers can use the knowledge, and Gray, Stolterman and Siegel (2014) in the two-way traffic between practice and research. Reeves (2015) shows why definitional work cannot close the question. The field's picture of science, he argues, has been constructing a disciplinary order rather than settling what HCI studies.

What would settle it is a measurement of value after publication. The CHI 2016 distribution records judgements at one moment for one venue. A study that followed named contributions of each type into citation and use by designers would test the uptake position directly. Hook and Lowgren (2012) supply a testable middle case. Their criterion that a strong concept be generative can be checked by asking whether a named concept is reused across domains by people who did not coin it.

Neither side owns a test of value. The type position rests on a submission instrument and on one conference's acceptance distribution. The paradigm position rests on a conceptual argument illustrated by a single cockpit example. The uptake position rests on a design proposal and on one interview study of working designers. Only the first carries numbers, and they record judgement before publication. While the dispute is open, a designer should name the contribution type and the community that will judge the work, then argue for the claim in that community's terms.

The strongest counter-argument: the type did predict the judgement

The strongest objection comes from the numbers in the paper the thesis rests on. If type were merely a vocabulary, the CHI 2016 acceptance rates would not have separated as they did. Empirical studies of people were accepted at 26.3 percent, and studies of system use at 23.5 percent, against a conference rate of 23.6 percent. Datasets were accepted at 16.7 percent, and they were the least submitted type at 1.3 percent of submissions (Wobbrock and Kientz, 2016).

Three things weaken it. The numbers record judgement before publication, and the authors warn that activity is not value. Method and theory contributions may be hard-won and then cited for years. The ordering does not follow evidential strength either. Essay or argument, the type defined as persuasion rather than evidence, was accepted at 21.6 percent, above method and theory. And a submission rate records what authors choose to send. Type does carry information; what the data do not show is that the type, rather than the community, is deciding.

Where the argument is heading

The direction is away from the label and towards the claim. Bodker (2006) describes the shift that forced the renegotiation, as second-wave HCI organised around work met a third wave in which technology spreads into homes and everyday life. Hook and Lowgren (2012) point to intermediate knowledge as the level at which design research will increasingly be judged. Shneiderman (2016) argues for a problem-driven programme combining applied with basic research and science with engineering and design. The inference that the contribution types are an institutional arrangement of the same kind is mine and not his. Reeves (2015) keeps the question open by asking for a reassessment of rigour. The next unit asks what such a claim rests on.

The sources this unit rests on

Each note is this unit's summary of what the cited document did and found, not a quotation. Every record was checked against the publisher's registered metadata on 1 November 2026.

• Wobbrock, J. O. and Kientz, J. A. (2016). Research contributions in human-computer interaction. doi:10.1145/2907069 Sets out seven contribution types and the test each faces, and reports their use at CHI 2016, which received 2,316 submissions and accepted 23.6 per cent of them. It is the unit's central source and the source of its strongest counter-evidence.
• Hollan, J., Hutchins, E. and Kirsh, D. (2000). Distributed cognition: toward a new foundation for human-computer interaction research. doi:10.1145/353485.353487 Proposes distributed cognition as a new foundation for HCI, arguing that interaction spans people, artefacts and networked environments rather than a desktop user. It changes the object of study rather than adding a finding.
• Rogers, Y. (2004). New theoretical approaches for human-computer interaction. doi:10.1002/aris.1440380103 Reviews the theories imported into HCI and finds that they broadened the field's discourse while having limited impact on the practice of interaction design, so a theoretical contribution answers to an audience beyond the conference.
• Sutcliffe, A. (2000). On the effective use and reuse of HCI knowledge. doi:10.1145/353485.353488 Finds direct application of cognitive theory to design limited by scalability problems, and proposes claims as a bridging representation carrying knowledge from theory to designers. Claims are grounded in specific scenarios, which limits their generality.
• Fallman, D. (2003). Design-oriented human-computer interaction. doi:10.1145/642611.642652 Argues that HCI became a design-oriented field whose basis did not keep pace, examines three accounts of what design is, and separates knowledge-generating design-oriented research from artefact-generating research-oriented design.
• Zimmerman, J., Forlizzi, J. and Evenson, S. (2007). Research through design as a method for interaction design research in HCI. doi:10.1145/1240624.1240704 Proposes research through design, in which designers make novel integrations of prior research to make the right thing, and formalises the model with four lenses and three examples. Design, it argues, earns its place on under-constrained problems.
• Hook, K. and Lowgren, J. (2012). Strong concepts: intermediate-level knowledge in interaction design research. doi:10.1145/2362364.2362371 Proposes strong concepts, an intermediate form of design knowledge more abstract than an instance and less general than a theory, and presents two, social navigation and seamfulness, against criteria including contestable, defensible and substantive.
• Gaver, W. (2012). What should we expect from research through design? doi:10.1145/2207676.2208538 Concludes that research through design yields theories that are provisional, contingent and aspirational, proposes treating theory as annotation of design examples and portfolios, and advises against convergence and standardisation.
• Gray, C. M., Stolterman, E. and Siegel, M. A. (2014). Reprioritizing the relationship between HCI research and practice: bubble-up and trickle-down effects. doi:10.1145/2598510.2598595 Interviews interaction designers about their use of design methods, including affinity diagramming and affordance, and builds a model of ideas bubbling up from practice and theory trickling down into it. A community that does not review papers thus helps confer value.
• Harrison, S., Tatar, D. and Sengers, P. (2007). The three paradigms of HCI. http://people.cs.vt.edu/~srh/Downloads/HCIJournalTheThreeParadigmsofHCI.pdf Argues that HCI contains three coexisting paradigms, human factors, classical cognitivism and a phenomenologically situated third, each a world view with its own criteria for knowledge. It runs one cockpit warning example through all three.
• Dourish, P. (2006). Implications for design. doi:10.1145/1124772.1124855 Argues that the demand for implications for design asks a study of one setting to yield requirements for systems development, and that the metric may fail to capture the value of an ethnographic investigation.
• Reeves, S. (2015). Human-computer interaction as science. doi:10.7146/aahcc.v1i1.21296 Traces cognitive science influences on HCI through early work on the design of input devices, and examines the scientific design space the author argues has animated the field since. It argues that the picture of science in HCI has been constructing a disciplinary order rather than settling the field's object.
• Bodker, S. (2006). When second wave HCI meets third wave challenges. doi:10.1145/1182475.1182476 Surveys second-wave HCI theory against a third wave in which technology spreads from the workplace into homes and everyday life and use is no longer one well-defined actor completing a task.
• Shneiderman, B. (2016). The new ABCs of research: achieving breakthrough collaborations. doi:10.1093/acprof:oso/9780198758839.001.0001 Argues for a problem-driven programme combining applied with basic research and science with engineering and design, treating those separations as obstacles rather than natural divisions. Reading the contribution types as an institutional arrangement is this unit's inference, not the book's claim.
• ACM SIGCHI (2024). Contributions to CHI. https://chi2024.acm.org/submission-guides/contributions-to-chi/ The CHI 2024 submission guide's own list of eight contribution types, from interface artefacts to validation and replication. It is the instrument now in use, and its types differ from the seven the unit describes.