Advanced qualitative and mixed-methods research

A qualitative study is a set of choices. Who to recruit, how long to stay, what counts as data, who does the analysing, and what makes a good answer. Each choice can be defended or defaulted, and the difference is the method. The default is whatever is nearest: the people who replied, the interviews that fitted a timetable, the themes that read well.

The contested question is how far a procedure can settle quality. Some methodologists reject systematic sampling altogether, as a positivist import that does not belong in interpretive work (Palinkas et al., 2015). Others offer one set of criteria that many traditions recognise, while leaving room for argument inside it (Tracy, 2010). A design has to be defended in the terms its tradition accepts.

A design is a set of choices you can be asked to defend

Purposeful sampling means choosing cases because they are information-rich, not because they are available (Palinkas et al., 2015). Maximum variation sampling looks for cases that differ, so a shared pattern has to survive the heterogeneity. Snowball sampling asks well-placed people who knows a lot about the topic, then follows their answers outward. A criterion sample takes everyone who meets a stated condition.

Palinkas and colleagues reviewed 28 implementation studies and found that 21 of them, or 75 per cent, used some form of criterion sampling. Their table adds a smaller item: convenience sampling, which recruits whoever is easiest to reach, is described as neither purposeful nor strategic. That is the honest name for the default.

A worked case from the same review. A study of an evidence-based practice (a treatment with research evidence behind it) recruits consultant trainers and programme leaders at each site. They hold the knowledge the question needs. It then asks those managers to name clinicians and administrative staff, which is snowball sampling in sequence. The sample was not dictated by a timetable.

What makes a choice defensible is the alternative you rejected. Write down who else you can recruit, why they answer a different question, and why yours matters.

Sampling for depth, and the honest account of saturation

Qualitative work aims at depth, where quantitative work aims at breadth and generalisability (Palinkas et al., 2015). The practical form of that aim is saturation: keep sampling until new data stop adding new information. The number is usually asserted and rarely demonstrated.

Hennink and Kaiser (2022) identified 23 articles that tested saturation using empirical data or statistical modelling. The studies using empirical data reached saturation within a narrow range of 9 to 17 interviews, or 4 to 8 focus group discussions. That range held particularly where the population was relatively homogeneous and the objectives narrowly defined. The outliers are the instructive part. Multi-country research and meta-themes (themes that gather lower-level themes under them) needed larger samples, and so did work seeking saturation of meaning rather than of codes.

The honest reading is conditional, and the authors say so. A range from other people's studies does not transfer to yours, because homogeneity and focus are properties of your design. Report where you stopped, the evidence for stopping, and the features of your sample that weaken the comparison.

Ethnography over months, and what long presence costs

Ethnography asks for time in a setting, and time buys what a single visit cannot. Blomberg and Karasti (2013) reflect on 25 years of ethnographic work in computer-supported cooperative work (CSCW, the study of how people work together with technology). They describe how the field site became a construct: not a bounded office, but a span of activity that moves across places.

Three costs are easy to miss. Access is a negotiation that continues, not a permission granted once. Field relations shape what people do while you are watching, so your presence belongs in the data. Long presence also creates obligations: consent obtained in March does not cover a conversation in September.

A constructed case, not a study result. An ethnographer spends six months with a hospital team and is treated as a colleague. In month five a clinician mentions, in passing, how staff work around a safety check. The sentence is the most valuable in the corpus, and it arrived unrecorded, after a shift. Recording it now changes the relationship. Noting that the claim exists, and writing about the decision, is the defensible move.

Blomberg and Karasti also record the running argument about what counts as proper ethnography, so the time you spent has to be described, not asserted.

Video and multimodal analysis, at the level of the sequence

Video records what audio loses: gaze, gesture, posture, and the way a body takes a turn without speaking (Heath, Hindmarsh and Luff, 2010). The unit of analysis is the sequence, not the person, which changes what a claim looks like. You cannot report that a user was confused. You can report that the second speaker waited after the first had finished, and that the next question narrowed the topic.

Heath and colleagues are candid that the difficulties begin before the recording does. Ethics committees ask what will happen to the material and how consent can be obtained for events that have not happened yet. Where to put the camera, what to frame, whether to collect in one intense period or successive phases: these are the practical questions.

The earliest film-based studies carry the argument. Bateson and Mead recorded Balinese behaviour as it happened, rather than staging the norms they wanted to see. Efron used frame-by-frame analysis to compare gesture across groups. Both moves remain the discipline: take the record as it comes, then slow it down.

A data session, where a team watches one clip repeatedly and argues about what it shows, is not a workshop ritual. Disagreement about the same thirty seconds is the method working.

Discourse and narrative analysis ask different things of one transcript

Discourse analysis is the close study of language and language use as evidence of aspects of society and social life (Taylor, 2013). Language is constitutive rather than a neutral channel, meanings are socially derived, situated, negotiated and co-constructed, and language use is a form of social practice. Taylor notes a fact beginners miss: there is no single discourse analysis, and one recognisable form of it is narrative analysis.

The two ask different questions of the same words. Discourse analysis asks what a version of events does, and whose interests it serves, by noticing what is treated as fact. Narrative analysis asks how events are ordered into a story with a teller, a plot and a point.

A constructed case, not a study result. A participant says the old system was fine until the team expanded. A discourse analyst notices that the account protects the speaker from blame, by attributing the problem to circumstances. A narrative analyst notices that the story has a before and an aftermath, and no active agent, and asks what the missing character is doing. Both readings can be checked against the transcript, and neither comes from a count of themes.

Mixed methods, and the integration point you name first

Combining strands is not a method. Fetters, Curry and Creswell (2013) describe integration at three levels. At the level of design there are three basic configurations: explanatory sequential, where numbers come first and interviews explain them; exploratory sequential, where interviews come first and produce the instrument; and convergent, where the two run in parallel. Four advanced frameworks extend this, including participatory and case study designs. At the level of method the strands connect, build, merge or embed. At the level of interpretation and reporting they come together through narrative, data transformation, or a joint display.

Fetters and colleagues also name the property that decides whether this worked: the fit of integration, the extent to which the two sets of findings cohere. A joint display is where that becomes visible. It puts a theme beside the number it explains, so a reader can see what the combination produced that neither strand produces alone.

Name the integration point before you collect. A construction, not a study result. A survey finds that two sites report very different error rates, and the interviews show that one site logs near misses and the other does not. Presented as two studies, the paper has a puzzle and no answer. Presented as one, with the explanation beside the number, the combination changes what the count means.

If the design has no named integration point, the honest label is two studies in one report.

Participatory and arts-based methods change who does the analysis

Participatory methods move the participant from subject to co-analyst, a design decision with consequences. Sanders and Stappers (2008) describe the shift from designing for users to designing with them, in which the person becomes the expert of their own experience. The researcher becomes a facilitator rather than a translator between the user and the designer.

Their framework separates levels of participation. They describe four levels of creativity in people's lives: doing, adapting, making and creating. The levels differ in the expertise and interest they demand. A method that asks everyone to create will exclude most participants. A method pitched at adapting includes more people and produces less invention.

The worked case they give is a three-dimensional toolkit used with nurses to design the ideal future patient room. The participants built a physical model, and it carried claims a questionnaire does not produce. The agency sat in the making, not in the answering.

The limit deserves stating plainly. Participation in analysis is often claimed and rarely documented, and co-designing an artefact is not the same activity as producing a research finding. Say which decisions the participants actually made, and keep the analysis answerable to you.

Quality criteria, and reflexivity as a document rather than a confession

Tracy (2010) answers the contested question by widening it. She separates the means of qualitative research from its ends, and proposes eight markers of quality: a worthy topic, rich rigour, sincerity, credibility, resonance, significant contribution, ethics, and meaningful coherence. The list is a vocabulary for an argument between reviewers, not a checklist that produces quality when applied.

Braun and Clarke (2021) answer from one tradition. Their book is the full account of reflexive thematic analysis, the version they developed, and it treats the analyst's interpretive work as part of the method. It covers the practical sequence, from familiarisation and coding to developing themes and writing the report, then theory, interpretation and quality.

Reflexivity is what makes the second answer more than a slogan, and its test is documentary. Not a paragraph about your position at the front of a report, but records another person can inspect: the questions you asked and the ones you did not, the reading of an extract you resisted, and what has to be true for a rival reading to win. Where there is one analyst, those records are the evidence that the analysis was examined at all.

The software organises, and the writing is where the claim becomes testable

Analytic software is a filing system with a search function. Coding inside it records where a code was applied and lets you retrieve every instance. It does not decide what deserves a code, and it cannot tell you when a theme has become a claim rather than a heading. Treat it as an organiser and keep the reasoning outside it.

Writing is the last design decision, and it is where qualitative work is usually lost or won. Blandford, Furniss and Makri (2016) begin from a gap. Qualitative methods matter in human-computer interaction for gathering requirements, for understanding situations of use, and for evaluating use in practice. Many texts cover them in the social sciences, few for HCI. Their book follows a study from planning through conduct to reporting.

Two disciplines carry a findings section. The first separates the participant's words from your claim, because a claim that paraphrases a quotation has not been made. The second reports disconfirming evidence: the case that does not fit, named and examined rather than dropped. Braun and Clarke (2021) place the writing of the report inside the analysis, not after it. The report is where you find out whether the analysis holds.

What follows takes these instruments to systems that adapt as they learn, where a claim must survive a design that changes while it is studied.