MSci project: scoping, ethics and method

A master's project is examined twice. The first examination is of the plan, and it happens before any data exist. Question, method, risks, ethics approval and a defended proposal are settled in advance, because a study that has not been approved cannot be run, and a question that cannot be answered in the time available no amount of later work rescues a question the calendar cannot answer. The plan is the deliverable, and it has to be real enough that somebody else could execute it.

A gap in the literature is not the same as an absence of reading

The fastest route to a weak project is a question built on a claim about your own reading. You notice that no study has compared a wrist-worn two-handed input device with a tablet, and write that this has not been studied. That sentence describes the limits of your search, not the state of the field. Sandberg and Alvesson (2011) reviewed 52 articles in organisation studies on how researchers construct research questions from existing literature, and reported that the most common move across different traditions is to spot a gap and formulate the question from it. Against gap spotting they set problematisation, which questions the assumptions a body of work takes for granted.

Gap spotting is not illegitimate, but it is weak when the gap is asserted rather than demonstrated: a gap that matters and a gap that exists for a reason look identical from outside. State the assumption your question rests on, then look for the study that would refute it.

A constructed case, not a study result. A student proposes that no one has measured how older adults use voice assistants for medication reminders. Read as a gap, the question survives a literature search. Read as a problematisation, the student asks why the field assumes that reminding is the difficult part, and finds that research on households locates the difficulty in how a reminder is negotiated between the person taking the medicine and whoever set the device up. The second reading yields a sharper question.

Any question then faces three tests. Feasibility asks whether you can reach the participants, build the apparatus and finish inside the calendar. Contribution asks which type of new knowledge the answer would be. Ethics asks whether the study can be approved at all, the test most often deferred and the one most likely to end a project.

Contribution has a type, and the type sets what you must show

Wobbrock and Kientz (2016) set out the forms that new knowledge takes in human-computer interaction: empirical, artefact, methodological, theoretical, dataset, survey and opinion contributions. An empirical contribution stands or falls on the importance of the finding and the soundness of its method. An artefact contribution is judged on the artefact and on the evidence about its use, not on the significance of a measured effect. A dataset contribution is judged on its quality and coverage. Naming the type in the proposal tells the panel what evidence will count.

The same paper reports the CHI 2016 distribution. Of 2,316 submissions, 44.0 per cent were empirical studies of system use. Empirical studies of people were 28.4 per cent of submissions and had the highest acceptance rate of any type at 26.3 per cent, against an overall acceptance rate of 23.6 per cent. The authors warn against reading activity as value: methodological and theoretical work may be harder to publish and more durable.

A project that releases a new interaction technique is making an artefact contribution. If its proposal claims an empirical contribution instead, it promises a finding, and a finding needs a design that could have produced the opposite result. Projects fail at the panel on exactly this mismatch: the artefact is real, the promised finding is not supported by the method proposed.

A literature review that positions rather than lists

A review has two jobs. The first is to establish what is known precisely enough that the contested question becomes visible. The second is to state the position the project will occupy, which means ending the review with the claim it establishes rather than with a summary of who wrote what.

Honesty about the quality of the evidence is part of the positioning. Cockburn, Gutwin and Dix (2018) reviewed earlier commentary on the role of evaluation in human-computer interaction research and on the practice of hypothesising after results are known. Their motivation is a concern about the credibility of experimental claims, not a claim that the field is broken. A review that treats every cited study as equally solid cannot support the argument the project needs, because the panel will ask which studies are strong and why.

One test for a draft: read only its final paragraph. If it does not state what is known, what is disputed, and what the project will do about the dispute, the review is still a list.

Method selection is a decision with rejected alternatives

A method is chosen against alternatives, and the alternatives have to be named. Greenberg and Buxton (2008) argued that usability evaluation, sound in its place, can mislead when it is applied by rule rather than by thought: run during early design it can suppress ideas that do not match current interface norms, and run to validate a prototype it can suggest a design is scientifically worthy without showing how anybody would adopt it. Their instruction is that the choice of evaluation methodology must arise from the research question. Cairns and Cox (2008) set out the wider repertoire, from experimental design and eyetracking to qualitative work and cognitive modelling.

A worked case. A project asks how a ward team comes to a shared understanding during shift handover. A controlled usability test would answer a different question: how quickly an individual completes a task against a specified measure. The proposed method is observation on the ward, with the handover recorded under ethics approval and the team interviewed afterwards. State the rejected alternative anyway: the panel will ask what was lost, and the honest answer is standardisation and comparability.

Pilot the parts that can fail, and read the results correctly

Leon, Davis and Kraemer (2011) describe the purpose of a pilot study as examining feasibility, and are firm about what a pilot is not. It is not a hypothesis test, so safety and effectiveness are not evaluated. Contrary to common practice, it does not supply a usable effect size for planning the main study, because an estimate from a small sample is too imprecise. Its feasibility results do not generalise beyond the inclusion and exclusion criteria of the pilot itself. What a pilot gives you is quantified recruitment, retention and assessment data, which is the evidence a panel wants about whether the main study is runnable.

A worked case. A project needs participants who use a screen reader daily, and the plan assumes twenty recruits in six weeks from a mailing list. A four-week pilot finds that the mailing produced almost no responses and that the respondents came through a specialist forum instead. The main study is re-planned around the route that works, and the recruitment figure in the proposal becomes defensible.

Report the pilot as a feasibility result. Reporting it as a small version of the main study invites the panel to read an underpowered comparison as a finding.

The ethics application is a schedule risk, not a formality

A weak ethics application delays a project more reliably than any technical failure, and the delay is measured in weeks because a committee meets on a cycle. The application is therefore written early and written fully, and scoped to the study that will actually be run rather than to a convenient version of it.

The Economic and Social Research Council (2025) states six principles: maximising benefit and minimising harm, respecting rights and dignity, voluntary and appropriately informed participation, integrity and transparency, clearly defined lines of responsibility, and independence with conflicts of interest made explicit. Bailey, Dittrich, Kenneally and Maughan (2012) summarise the Menlo Report, which extends the Belmont principles for research on information and communication technology and adds a fourth, respect for law and public interest, with legal due diligence, transparent methods and accountability for action. Its most useful point for an HCI project is about who counts: the framework covers people affected by the research who are not its subjects.

A worked case. A study records participants' screens while they complete an online banking task. The recording captures the participant, but it can also capture messages and names belonging to other people who never consented. The honest application names that third-party exposure, says how the frames will be redacted, sets a retention period, and states who will hold the files. Omitting any of that is what sends the application back.

Preregister what can be preregistered, and account honestly for what cannot

Nosek, Ebersole, DeHaven and Mellor (2018) draw the line that matters. Generating a hypothesis from existing observations and testing one on new observations are different acts, and ordinary biases such as hindsight bias make the difference hard to keep. Preregistration, that is, defining the research questions and analysis plan before observing the outcomes, records which analyses were predictions and which were postdictions. The authors are careful about their own case: they call the evidence for preregistration's superiority for reproducibility theoretically strong and supported by some correlational evidence, but say experimental evidence establishing it is not yet sufficient.

Chambers and Tzavella (2022) describe the stronger instrument, the Registered Report, in which a proposal is peer reviewed and given in-principle acceptance before the research is undertaken, so publication no longer depends on the results. They are explicit that Registered Reports are not a universal solution for irreproducibility. Cockburn, Gutwin and Dix (2018) take the question to the field, discussing how preregistration could be adapted to the distinctive characteristics of HCI research, where prototyping and exploration are legitimate parts of the method.

A worked case. A project whose main study is a controlled comparison of two input techniques can preregister its hypotheses, primary measure and analysis, and can say which comparisons are exploratory. A design study producing an artefact cannot wholly preregister, and should not pretend to. The honest account labels each analysis as planned or emerged, and says what the pilot changed. The second project is not the weaker one. The dishonest account is.

The plan, the register and the supervision agreement

A plan is milestones with dates and named deliverables, and a milestone that names a phase is not a milestone. Write the date by which the ethics application is submitted, the date by which the pilot is analysed, the date by which a full draft exists. Working backwards from the submission date is the only reliable way to find that a two-month ethics queue makes a planned start impossible.

A risk register lists what could stop the project, how likely each risk is, how much damage it would do, what would be done about it, and who owns it. The risks worth writing down are recruitment, access to a site or a dataset, ethics delay, a device that behaves differently in the field, and a skill still to be learned before analysis starts. A register with no entry for ethics delay has not been thought about.

The supervision agreement is a short document about meetings and feedback. Agree the meeting frequency, the channel for questions between meetings, how far ahead a draft must arrive to be read, and what the supervisor will and will not read. A worked case: a student sends a chapter the evening before the meeting and receives comments on spelling rather than on the argument. The agreement exists so that the argument gets read.

What the defence has to establish

The proposal defence is not a test of how much you have read. It is a test of whether the second half can be run. Expect the panel to ask what the study would show if the results went the other way, which alternative method was rejected and what that choice cost, who the participants are and how they will be reached, what the ethics application says about data that is not yours, which analyses are confirmatory and which exploratory, and what happens if recruitment fails. A plan that answers those questions has already done most of the work of running the study, and it hands straight into the delivery, analysis and defence that follow.