What the project promises: contribution types and the evidence a panel accepts

A proposal makes two promises. The first is about its subject; the second is about the knowledge, and it says what kind of new thing the answer will be. This second of ten units argues that every project promises one kind of new knowledge, and the third unit asks how such a claim is positioned in the literature. That kind fixes the evidence a panel will accept. A panel judges a proposal that mislabels an artefact project as an empirical one against a standard its method was never designed to meet.

The subject can be revised after the pilot. The kind of knowledge is what the panel will hold the project to.

Wobbrock and Kientz (2016) give the second promise its name. Every contribution in human-computer interaction is one of seven types, each judged on its own terms:

• empirical;
• artefact;
• methodological;
• theoretical;
• dataset;
• survey;
• opinion.

An empirical contribution stands on the importance of its finding and the soundness of its method. An artefact contribution stands on the artefact and the evidence about its use, rather than on a measured effect. A dataset contribution stands on quality and coverage.

A taxonomy describes articles, and a project needs a position

Wobbrock and Kientz (2016) also report how CHI 2016 used the scheme. Of the 2,316 submissions, empirical studies of system use were 44.0 per cent. The dataset contribution was both the least submitted and the least accepted type.

Fallman (2008) supplies that position in the interaction design research triangle of design practice, design studies and design exploration. Those are combinable modes rather than competing paradigms, so a student can state where the work sits.

Oulasvirta and Hornbaek (2016) offer something that cuts across the types. Drawing on the philosophy of Larry Laudan, they argue that most HCI work addresses one of three kinds of problem: empirical, conceptual and constructive. They propose problem-solving capacity as a common currency for comparing research of different kinds. That currency is the strongest challenge to this unit's thesis.

An artefact is a claim about a problem, not a failed experiment

Where the contribution is an artefact, the panel is not waiting for a measured effect. March and Smith (1995) distinguished design research from natural-science research and organised it around four products: constructs, models, methods and instantiations. A panel judges design research on usefulness and on performance against requirements, among them access for a user who cannot use a mouse, not on the truth of a proposition.

Hevner, March, Park and Ram (2004) set out design science as the creation and evaluation of artefacts that extend human and organisational capabilities, complementary to the behavioural-science paradigm. Their guidelines require that an artefact address a relevant problem and be rigorously evaluated.

Peffers, Tuunanen, Rothenberger and Chatterjee (2007) turn the paradigm into a process of six activities, from problem identification through design, demonstration and evaluation to communication. Four case studies demonstrate the process. A proposal claiming an artefact contribution must plan its milestones against those activities. The four cases they work the method through are a database supporting health assessment methods, a software reuse measure, an Internet video telephony application and an information systems planning method.

Chatterjee, Bhattacherjee and Gilb (2024) extend the typology. They separate knowledge about the artefact from knowledge about the design process and about the problem domain. They argue that making the type explicit helps a project defend what it contributes.

What research through design can promise, and what it cannot

Zimmerman and Forlizzi (2014) describe research through design, in which making an artefact is the means of inquiry. They set out what it contributes: knowledge about the design process and about the problem a design addresses. Such knowledge is provisional and situated, judged by its ability to reframe a problem rather than by statistical generalisation.

Gaver (2012) concludes from two theories in the philosophy of science that research through design produces provisional, contingent and aspirational theories rather than universal ones. It should be judged by its ability to generate new design possibilities, rather than by explanatory standards. A project that says so is asking to be judged on a stated and different ground, not to be excused rigour.

Intermediate-level knowledge: the type the taxonomy stretches to hold

Hook and Lowgren (2012) propose strong concepts as intermediate-level design knowledge, more abstracted than a single design instance but less general than a theory. The concepts are generative across use situations, and the authors validate them by theorising the design, examining the instance and probing transferability. The contribution is neither artefact nor theory, so claiming either invites judging by the wrong standard. The two concepts they work through are social navigation and seamfulness, each pitched at a level that travels across use situations rather than describing a single design.

Theory is a contribution type with its own standard

Rogers (2004) reviewed the theoretical approaches that have shaped human-computer interaction. He argued that theory in the field has largely been borrowed and applied rather than generated within it. That thinness is why a project's theory usually arrives from outside and must be argued for.

Rogers (2012) distinguishes the uses of theory across the classical, modern and contemporary periods, from prediction to framing. He argues that a project should say which use it intends. Sutcliffe (2000) adds the test: direct application of theory to design practice has been limited. Bridging models that translate theory into guidance are needed instead.

What the bibliometric record shows, and what it does not

Wania, Atwood and McCain (2006) used author cocitation analysis to examine how the design and evaluation communities in HCI relate. They found the relationship between them unclear, and resolved the published literature into seven clusters rather than one connected body. Their result is about communities of authors rather than about individual proposals, so it does not measure how often a study designs without evaluating. This unit reads it as context for the mismatch it names, not as evidence that the mismatch is common. The seven clusters are design theory and complexity, design rationale, cognitive theories and models, cognitive engineering, computer-supported cooperative work, participatory design, and user-centred design.

Contested question: does the type fix the standard, or only the paperwork?

The first position is this unit's: each type carries its own criteria, and a project is judged against the criteria of the type it claims. Wobbrock and Kientz (2016), March and Smith (1995), Hevner and colleagues (2004), Peffers and colleagues (2007) and Zimmerman and Forlizzi (2014) each supply a version of it.

The second is Oulasvirta and Hornbaek's (2016). If problem-solving capacity is a common currency, empirical, conceptual and constructive work can be compared on what each enables.

The third and strongest is that the standards are irreducibly plural. Gaver (2012) and Zimmerman and Forlizzi (2014) argue that research through design is judged by its ability to generate new design possibilities and to reframe problems. Hook and Lowgren (2012) add an intermediate level the seven types do not name. No single type then fixes the evidence, and an artefact project labelled empirical is mismatched to one standard among several.

No study settles the dispute, because none tests whether the claimed type predicts how a panel judges a proposal. Wobbrock and Kientz (2016) report acceptance rates by type for CHI 2016 but do not control for quality. That distribution shows what was submitted rather than whether the type fixed the standard. What would settle it is a matched design: two versions of one proposal, differing only in the type claimed, judged blind with quality held constant. Wania, Atwood and McCain (2006) find the relationship between the design and evaluation communities unclear. That is consistent with this thesis, but it does not show that the type is what a panel uses.

Read by kind, the unit's position rests on a taxonomy, a count of acceptance rates and a set of design-science guidelines, while its sharpest challenge rests on an argument from the philosophy of science. Neither side tests what a panel does with a proposal. While the dispute is open, a student should name the contribution type, state the criterion that type is judged by, and where the work is hybrid name the second criterion it must also meet. The strongest objection is that Oulasvirta and Hornbaek (2016) may be right that capacity, not label, is what a panel weighs, since their common currency compares empirical, conceptual and constructive work by what each enables.

Where the argument is heading

The direction of travel is towards naming the promise in the proposal itself. Chatterjee, Bhattacherjee and Gilb (2024) separate knowledge about the artefact, the process and the problem. Oulasvirta and Hornbaek (2016) propose a currency that crosses the types. Rogers (2012) sets out how a project can say whether its theory predicts or frames. A panel is increasingly asked not only what a project will study but what kind of knowledge it will produce.

The sources this unit rests on

Each note states what the work did, what it found and how it bears on the argument. Every identifier was resolved on 31 October 2026.

• Wobbrock, J. O. and Kientz, J. A. (2016). Research contributions in human-computer interaction. interactions. doi:10.1145/2907069 Sets out seven contribution types and how each is judged, and reports the CHI 2016 distribution of 2,316 submissions, of which 44.0 per cent were empirical studies of system use and dataset contributions were least accepted. Carries the unit's anchor claim.
• Hevner, A. R., March, S. T., Park, J. and Ram, S. (2004). Design science in information systems research. MIS Quarterly. doi:10.2307/25148625 Sets out design science as the creation and evaluation of artefacts that extend human and organisational capabilities, complementary to the behavioural-science paradigm, and requires that an artefact address a relevant problem and be rigorously evaluated. Supplies the criteria by which an artefact contribution is judged.
• Oulasvirta, A. and Hornbaek, K. (2016). HCI research as problem-solving. CHI Conference on Human Factors in Computing Systems. doi:10.1145/2858036.2858283 Argues, drawing on the philosophy of Larry Laudan, that most HCI work addresses an empirical, conceptual or constructive problem, and proposes problem-solving capacity as a common currency for comparing research of different kinds. The unit's strongest counter-position.
• Fallman, D. (2008). The interaction design research triangle of design practice, design studies, and design exploration. Design Issues. doi:10.1162/desi.2008.24.3.4 Proposes that interaction design research operates in three combinable modes, design practice, design studies and design exploration, which are positions in a triangle rather than competing paradigms. Gives a proposal a way to locate itself among design-oriented, study-oriented and explorative claims.
• March, S. T. and Smith, G. F. (1995). Design and natural science research on information technology. Decision Support Systems. doi:10.1016/0167-9236(94)00041-2 Distinguishes design research from natural-science research and organises design research around four products, constructs, models, methods and instantiations, evaluated on different grounds. Argues design research is judged on usefulness and performance against requirements rather than on truth.
• Peffers, K., Tuunanen, T., Rothenberger, M. A. and Chatterjee, S. (2007). A design science research methodology for information systems research. Journal of Management Information Systems. doi:10.2753/mis0742-1222240302 Presents a design-science methodology with six activities, from problem identification through design, demonstration and evaluation to communication, demonstrated across four case studies. Supplies the process an artefact proposal must plan its milestones against.
• Chatterjee, S., Bhattacherjee, A. and Gilb, T. (2024). A typology of knowledge creation in design science research projects. Lecture Notes in Computer Science. doi:10.1007/978-3-031-61175-9_10 Proposes a typology of knowledge creation in design science research that separates knowledge about the artefact from knowledge about the design process and about the problem domain. Argues that making the type explicit helps a project defend what it contributes.
• Zimmerman, J. and Forlizzi, J. (2014). Research through design in HCI. Springer eBooks. doi:10.1007/978-1-4939-0378-8_8 Describes research through design, in which making an artefact is the means of inquiry, and sets out what it contributes, including knowledge about the design process and about the problem a design addresses. Argues such knowledge is provisional and judged by its ability to reframe a problem.
• Gaver, W. W. (2012). What should we expect from research through design? CHI Conference on Human Factors in Computing Systems. doi:10.1145/2207676.2208538 Argues, drawing on two theories from the philosophy of science, that research through design produces provisional, contingent and aspirational theories rather than universal ones. Concludes it should be judged by its ability to generate new design possibilities rather than by explanatory standards.
• Hook, K. and Lowgren, J. (2012). Strong concepts. ACM Transactions on Computer-Human Interaction. doi:10.1145/2362364.2362371 Proposes strong concepts as intermediate-level design knowledge, more abstracted than a single instance but less general than a theory, generative and validated by theorising the design, examining the instance and probing transferability. Carries the unit's awkward case, neither artefact nor theory.
• Rogers, Y. (2004). New theoretical approaches for human-computer interaction. Annual Review of Information Science and Technology. doi:10.1002/aris.1440380103 Reviews the theoretical approaches that have shaped HCI and argues that theory in the field has largely been borrowed and applied rather than generated within it. Explains why a project's theory usually arrives from outside the field.
• Rogers, Y. (2012). HCI theory: classical, modern, and contemporary. Synthesis Lectures on Human-Centered Informatics. doi:10.2200/s00418ed1v01y201205hci014 Surveys HCI theory across three periods, organising the theories by the role they play and the assumptions they make. Argues that theory is used in different ways, from prediction to framing, and that a project should say which use it intends.
• Sutcliffe, A. (2000). On the effective use and reuse of HCI knowledge. ACM Transactions on Computer-Human Interaction. doi:10.1145/353485.353488 Reviews the role of theory in HCI design and argues that direct application to design practice has been limited, so bridging models which translate theory into design guidance are needed. Shows a theoretical contribution is judged by its use as well as by its truth.
• Wania, C. E., Atwood, M. E. and McCain, K. W. (2006). How do design and evaluation interrelate in HCI research? Proceedings of the 6th conference on Designing Interactive Systems. doi:10.1145/1142405.1142421 Classifies HCI research by how the design and evaluation communities relate, using author cocitation analysis over a sample of published work, and finds the relationship between them unclear, with the literature resolving into seven clusters. Carries the unit's caution that the field's own record does not measure study-level mismatch, rather than an empirical warrant that the mismatch is common.