Research frontiers in HCI
HCI4101 · 15 credits · Level 7
How the field produces knowledge, how to read a paper as an argument, and where the current frontier actually is.
Ten units
- 1What counts as a contribution: seven types, and the arguments about themThe field's seven contribution types are a working vocabulary rather than a settled test, and the debates that followed them, over research through design, intermediate-level knowledge and the traffic between research and practice, show that what counts as a contribution is decided by a community's judgement of value rather than by the type alone, so a researcher who wants to be read must argue for the kind of claim they are making and not merely for its soundness.
- 2A paper is a claim, its evidence and the inference between themThe distance between a paper's claim and its evidence is not a rhetorical accident but a systematic product of how results are analysed and reported, so a reader who cannot re-derive the inference from the data cannot tell a warranted claim from an inflated one, and the routine misreading of statistics is the most common way that gap is opened.
- 3Venues are communities of judgement, and their judgements are studyableA venue is not a container for a finding but the community that decides what a finding is worth, and the peer-review literature shows that this judgement is moved by prestige, by blinding and by the incentives around publication, so reading a paper means reading the community that accepted it as much as reading the paper.
- 4Most findings rest on one study, and one study is weak evidenceA finding confirmed once is a hypothesis with a result attached, and the field's own audits show replication to be rare in HCI and unreliable in the neighbouring sciences, so the credibility of a claim should be a function of how many independent lines of evidence support it rather than of how confidently it is stated.
- 5Researcher degrees of freedom turn noise into findingsSignificance is partly a property of the analyst's choices rather than of the world, because the flexibility to select measures, subsets and stopping points lets an absent effect be presented as a present one, so the strongest defence is not a better statistic but the fixing and recording of the decisions before the data are seen.
- 6Open science answers the failures, at a pricePreregistration, registered reports and open data address the credibility failures by moving decisions ahead of the data, and the evidence that they work is real but partial, so the honest position is to label planned analysis and exploratory finding separately and to accept that the remedy buys credibility with freedom.
- 7A systematic review is a method, not a reading listA synthesis is only as trustworthy as the protocol that produced it, and the reporting standards that now govern systematic reviews make the search, the screening and the appraisal auditable, but the method transfers imperfectly into HCI, so the field's reviews range from rigorous evidence to a reading list with a flowchart attached.
- 8A review organises a field rather than listing itThe reviews that last are the ones whose organising idea a later reader can test, which makes a review a theoretical contribution in its own right, and the typologies of review types show that choosing a review genre is choosing what kind of claim the review is able to make.
- 9Generative models as interaction partners: the frontier and its evidence problemThe generative-model frontier has changed what interaction researchers build faster than it has produced evidence about whether the results help, so the field's guidelines and design principles are syntheses of practice rather than tests of outcomes, and the evaluation methods that would settle the question are still being invented.
- 10Auditing a frontier claim: separating evidence from projectionA frontier claim mixes measurement with hope, and the audits that matter are the ones that put a number on a material cost, a measured distortion of a study in the field, or the intellectual inheritance of a vision, so a student who can say which part of a claimed frontier is evidenced and which is projection has learned the module's method and not only its literature.
10 of 10 units written. Rows marked in preparation have no body yet.
Specification
| Assessment | Weight | Type |
|---|---|---|
| Critical paper analyses across the term | 30% | coursework |
| Bounded systematic review | 50% | coursework |
| Frontier position paper with evidence audit | 20% | coursework |
What you will be able to do
- Place a paper by contribution type and evaluate whether its evidence supports its claim.
- Identify at least three ways a published study could be strengthened and say which matters most.
- Conduct a bounded systematic search and report it with its limitations.
- Write a literature review that takes a position rather than summarising.
- State the evidence base behind a claimed frontier and separate it from the projection.
The reading this module rests on
- Wobbrock, J. and Kientz, J. (2016). Research contributions in human-computer interaction. Interactions 23(3), 38-44. The clearest statement of what counts as a contribution in this field.
- Munafo, M. et al. (2017). A manifesto for reproducible science. Nature Human Behaviour 1:0021. The remedies, stated compactly.
- Page, M. et al. (2021). The PRISMA 2020 statement. BMJ 372:n71. The reporting standard for the systematic review assessment.
- Rogers, Y. (2012). HCI Theory: Classical, Modern and Contemporary. Morgan and Claypool. Re-read for the theory map at master's level.
Research frontiers in human-computer interaction
A research paper is an argument, and the argument is usually smaller and more fragile than the abstract suggests. The abstract states what the authors believe they have shown. Between that claim and the data sits a chain of inference, and the chain has links that can break. Reading a paper well means finding those links and testing them one at a time.
That skill turns a consumer of the literature into a participant in it. A participant can place a new paper against what already exists, borrow the parts that hold, and repair the parts that do not. The work needs three things: a vocabulary for what a contribution is, an account of how the field judges one, and a method for organising what a field has already found.
A paper is a claim, its evidence, and the step between them
Separate three things whenever you read. The claim is the one sentence the authors want you to believe. The evidence is what they observed: what they ran, on whom, and what came out. The inference is the step that joins the two, and it is almost always wider than the evidence fills.
Suppose a paper reports that a new pointing technique cut target-selection time by about a tenth across 18 participants, all of them university students. The claim is that the technique is faster than the standard one. The evidence is 18 students on one task in one room. The inference is the gap between them: that the result holds for the users the technique is for, in the places they work, at the sizes and speeds they meet. The gap does not make the paper wrong. It tells you what the paper has earned and what the next study must test.
Sampling widens the same gap. A study of 20 participants is often written up as though it could detect a small difference. With 20 people, a small effect usually stays invisible, and a large estimate drawn from a small sample tends to exaggerate the truth. This is not a fault peculiar to interaction research. It is a property of sampling, and it is one reason the field's many small studies must be read with care.
Seven kinds of contribution, and why the kind decides the test
Wobbrock and Kientz name seven contribution types in the field: empirical, artifact, methodological, theoretical, benchmark or dataset, survey, and opinion (Wobbrock and Kientz, 2016). A single paper can make more than one. The list is a way to place a paper before judging it, because each type is held to a different test.
An empirical contribution is a new finding from systematically gathered data, and it is judged on whether the method is sound and the finding matters. An artifact is an invention, such as a system, technique or design, and it is judged on what it makes possible. A methodological contribution is a new way to build or to measure. A theoretical contribution explains or predicts. A benchmark or dataset gives the field something to compare against. A survey maps what is known. An opinion piece argues a position, and it is judged on the strength of the argument rather than on data.
The practical point is that the pointing-technique paper is an artifact contribution and an empirical one at the same time. A reviewer reading it as artifact asks what new possibility it opens. A reviewer reading it as empirical asks for the effect size, the statistics and the participants. The same paper faces two examinations, and a strong paper answers both.
Planning begins here, not after the study is run. Choose the contribution type first, because the type tells you what evidence will be demanded and what the related work must establish.
Venues are communities, and each community has its own test
The field publishes in conference proceedings and in journals, and the two work differently. CHI, the ACM Conference on Human Factors in Computing Systems, and CSCW, the conference on Computer-Supported Cooperative Work and Social Computing, are annual conferences. UIST, the ACM Symposium on User Interface Software and Technology, and DIS, the ACM Conference on Designing Interactive Systems, are conferences in the same family. TOCHI, the ACM Transactions on Computer-Human Interaction, and IJHCS, the International Journal of Human-Computer Studies, are journals. Conferences generally review in one cycle, sometimes with an author response. Journals review in rounds and allow revision. A single review rewards work that is complete and clear at the first reading. A journal round rewards work that can be argued with and improved.
One case, read in three communities. A team builds a tool that helps blind users label the objects in a kitchen. Read at UIST, the contribution is the technique and the system, and the paper must show that they work. Read at CHI, the contribution is often the empirical study of how people use them, and the paper must show what was learned about the users. Read at CSCW, the question shifts again, to the household and the shared work of keeping a kitchen usable, and an interview study may carry the paper on its own. The artifact has not changed. The community has, and it wants different evidence.
Read the venue before you read the paper. The norms are not written in one place. They are visible in what the venue has published over the last few years.
Most findings rest on one study, and one study is weak evidence
A finding confirmed once is a hypothesis with a result attached. Hornbaek, Sander, Bargas-Avila and Grue Simonsen examined four publication outlets and 891 papers, and they found that 3% attempted to replicate an earlier result (Hornbaek et al., 2014). The replications that existed usually treated the exercise as a decision to confirm or not. They rarely compared effect sizes with the original study. Many papers that were not framed as replications could have corroborated earlier work with a little more data or a different analysis.
Two failures make the problem worse. p-hacking is trying analyses until one gives a significant result, and reporting only that one. HARKing is presenting a hypothesis formed after seeing the results as though it had been stated in advance. Munafo and colleagues name these practices among those that inflate the share of published findings that are false, and they argue that the remedies belong to researchers, journals and funders together (Munafo et al., 2017).
A concrete case. A group runs a study on error rates for two input methods and the prediction fails. They then look at the time per item, find a difference in one subset, and write the paper around that subset. Nothing is fabricated, and the paper still misleads, because the reader cannot tell which analyses were planned and which were chosen after the fact.
Preregistration and open materials answer the failures, at a price
Preregistration means writing down the question, the design and the analysis before the data arrive, and lodging that record where a reader can find it. Its strongest form is the registered report, in which a journal reviews the plan and can accept the paper before the result is known (Munafo et al., 2017). Open materials and data let a reader check the analysis rather than trust it. Blinding the analyst to the condition removes one way for expectation to shape a decision.
None of this is free. Preregistration reduces the freedom to follow an interesting lead found in the data, and genuinely exploratory work does not fit the format. The honest position is to label which part of a study was planned and which was exploratory, and to give the exploratory finding the weaker status it deserves.
A systematic review is a method, not a reading list
A systematic review answers a question by finding, selecting and appraising the studies that bear on it, and it reports every step so that another person could repeat the search. PRISMA 2020, the preferred reporting items for systematic reviews and meta-analyses, is the checklist that disciplines the reporting. It lists 27 items for the review, with a further short checklist for the abstract and flow diagrams for new and updated reviews (Page et al., 2021).
The discipline is in the order of the decisions. Write the question and the protocol first. Search, then screen against stated criteria, and record how many studies fell at each stage. Appraise the risk of bias in each included study. Then synthesise, which means combining results formally in a meta-analysis when the studies measure the same thing, and describing them in words when they do not. A meta-analysis that pools studies measuring different constructs produces a number that means very little.
A bounded version suits a single project. Ask one question about one interface feature, search two databases with a stated string, and report the count at each stage. The value is the honesty of the count, not the size of the harvest.
Writing a review that organises a field rather than listing it
A list of papers tells the reader what exists. A review tells the reader how to see the field. The difference is a position. State what the field agrees on, where the evidence points in two directions, and what remains unasked. Group papers by the argument they support, not by their date or their authors. Where a claim rests on one small study, say so, and let that shape the conclusion.
The reviews that last are the ones whose organising idea a later reader can test. If a review claims that a field has converged on a design rule, a reader should be able to check whether new studies keep confirming it. That is what makes the review a contribution rather than a summary.
Auditing a frontier: telling evidence from projection
Each frontier claim of the moment mixes measurement with hope, and the skill is to take them apart.
Generative models as interaction partners. Weisz and colleagues derive six design principles for generative applications from a literature review, practitioner feedback, and their use in two real applications (Weisz et al., 2024). The principles are a useful vocabulary. They are also a synthesis of practice, not a test of outcomes, so they do not show whether a generative interface helps anyone.
Spatial and wearable computing. Ens and colleagues gathered 17 research challenges for immersive analytics from 24 international experts (Ens et al., 2021). A challenge list is evidence about where a community believes it is stuck. It is not evidence that the technology will deliver, and the authors present it to coordinate work rather than to predict.
Sustainability and the material footprint. This frontier carries the firmest numbers. Peer-reviewed studies estimate the current share of global greenhouse gas emissions from information and communication technology at 1.8 to 2.8%, and Freitag and colleagues argue that adjusting for missing supply-chain pathways raises the figure to between 2.1 and 3.9% (Freitag et al., 2021). They also find that sector pledges are not ambitious enough for climate targets. This is the case for treating material cost as a design constraint.
Decolonial and more-than-human design. Dourish and Mainwaring argue that ubiquitous computing carries a colonial impulse in its universalising ambition, and they set out what it would take to decolonise its method (Dourish and Mainwaring, 2012). Frauenberger goes further and asks whether the field should move past the human as its centre, drawing on post-humanist theory (Frauenberger, 2020). Both are position papers. They change the questions a designer asks, and neither reports a study.
Long-term deployment. Brown, Reeves and Sherwood ran a study of field trials and found that participants adjust their behaviour to fit what they think the study wants, and that a result can rest on the insights of a few participants (Brown et al., 2011). This is the honest core of the deployment turn: the world is the right place to test a system, and the world also distorts the test.
What follows from reading like this
You now have the instruments: the contribution types, the argument structure, the rigour failures and their remedies, and the audit that separates a frontier's evidence from its projection. The next step is to point them at studies that generate their findings in words rather than numbers, where sampling, analysis and the researcher's own position carry the weight that statistics carries here.