The interview: talk as evidence and the limits of asking

The interview is the default instrument of user research, and it is usually defended by an assumption nobody states. The participant's account reports something that happened, and the interviewer collects it undistorted. This fourth of the ten units argues the opposite. An interview is evidence about what people say and mean rather than about what they did. The self-report literature shows that direct questions about causes and future behaviour produce confident answers that need not be true. Talk must therefore be treated as the object of study. The conditions under which an account was produced become part of the finding rather than noise to be minimised.

An interview does not open a window on a life. It produces a conversation, and the shape of that conversation is part of the evidence rather than a flaw in it.

There is no single thing called the interview

Before any question of technique, a study has to say which tradition it is in. The interview is several incompatible methods wearing one name. Gubrium and Holstein (2001) survey the field from structured survey interviewing through life history to poststructural approaches. They frame the interview as an active social encounter in which both parties construct the account. Roulston (2010) sets the competing theories of what an interview produces against each other, and treats the interviewer's reflexivity as part of the method. Kvale and Brinkmann (2009) organise the enterprise as a craft with stages, from thematising the research question to analysing and reporting. They argue that quality is settled at the design stage and cannot be recovered later in analysis. The traditions are not interchangeable. A structured survey interview fixes the answer set before the encounter. A life-history interview treats the shape of the account as the finding. A study that runs one while claiming the other has misdescribed its own instrument.

Asking for a cause asks for a theory the speaker may not hold

The strongest evidence against treating talk as a window on behaviour concerns explanations rather than descriptions. Nisbett and Wilson (1977) reviewed the experimental literature. They found that people have little or no direct introspective access to higher-order cognitive processes. The distinction covers unawareness of a stimulus, of the response and of the link between them. Their proposal is that reports of process draw on a priori causal theories rather than an internal record. The answer is a plausible story assembled from what the speaker believes about people like them. Wilson and Dunn (2004) concluded that introspection cannot provide a direct pipeline to mental processes. Indirect routes, such as observing one's own behaviour, can improve self-knowledge. Their own stocking study is the demonstration. Fifty-two passers-by were approached under the guise of a consumer survey. They were asked to judge which of four pairs of nylon stockings was the best quality. The four pairs were identical. They chose the rightmost pair about four times as often as the leftmost. None of them mentioned position when asked why. Virtually all denied it when asked directly, citing the quality or the texture of the fabric instead. This is a prediction rather than a warning about dishonesty. Asked why they abandoned a form, a participant will answer coherently, and the answer may correspond to nothing that governed the behaviour.

Verbal reports are data when the mechanism behind them is respected

The counterweight is not a defence of introspection but a theory of when reports work. Ericsson and Simon (1980) argue that verbal reports are data whose validity depends on the mechanisms generating them. They offer a model in which a participant thinking aloud verbalises information currently heeded in short-term memory. On that account a report is reliable where the process is heeded and held long enough to be spoken. It is unreliable where the process is automatic or already run. The two literatures are not in contradiction once their scope is stated. Concurrent talk about a task sits where Ericsson and Simon mark it defensible. A retrospective explanation of why something was done sits where Nisbett and Wilson mark it reconstruction. Keep the report close to the task. Ask about the task, not its cause. Do not ask people to forecast what they will do.

The account is made in the interview, not merely delivered there

If talk is the object, the interaction that produced it has to be analysed. Rapley (2001) makes the case by close reading. He shows how the interviewee and the interviewer each build themselves as a certain kind of person in relation to the topic. His argument is that interviews are inherently social encounters. Attention should therefore fall on how accounts are produced, including the interviewer's own central role, rather than on talk as a direct report of attitudes. Potter and Hepburn (2005) separate contingent problems from necessary ones. A contingent problem a researcher can manage is the deletion of the interviewer, which leaves a published analysis whose questions never appear. Another is the failure to treat interviews as interaction. A necessary problem is the way social science agendas and categories flood the encounter. They treat the contingent problems as tractable once interview talk is the phenomenon under study rather than a pointer at something else. Denzin (2001) pushes further. He locates the reflexive interview within what he calls the cinematic-interview society. He describes it as a performance, which makes reflexivity a requirement of validity.

Ubiquity is not validity, and the medium is part of the finding

Atkinson and Silverman (1997) trace how the interview became the central instrument of social research. They argue that it has become a cultural form in which a self is produced for an audience. The account is partly a product of the genre rather than a sample of the person. Nunkoosing (2005) writes the direct version of the objection. He examines power and resistance, the distinction between truth and authenticity, and the difficulty of consent. He argues that the interview's popularity rests on assumptions about transparent talk that are rarely defended. An interview can establish what a person says, how they say it and the conditions of the encounter. It cannot on its own establish what they did.

The same argument covers the medium. Interviewing has moved away from the shared room, and the move is not neutral. Lo Iacono, Symonds and Brown (2016) draw on studies of dance and wayfinding that needed international and purposive samples. They weigh the reach and low cost of voice-over-internet interviewing against its effects on rapport, non-verbal cues and ethics. Their conclusion is that the medium cannot replace face-to-face interaction but works as a viable complement. The reach is uneven: a participant who cannot easily travel, through disability, age or distance, can take part from home, and one with no private room or a slow link cannot. The medium and the relationship are conditions of the data rather than a channel to it.

Contested question: does the account's construction weaken the interview or redefine it?

The literature agrees that an interview produces talk and disagrees sharply about what follows. One position holds that it is a knowledge-producing conversation whose products are usable when the researcher is reflexive and the technique sound. Kvale and Brinkmann (2009) place the guarantee at the design stage. Roulston (2010) treats the competing accounts as choices a reflective researcher makes. Ericsson and Simon (1980) supply the conditions under which a report is trustworthy. The opposing position holds that construction disqualifies the interview for many purposes. Atkinson and Silverman (1997) argue that ubiquity is not validity, and Nunkoosing (2005) argues the assumptions of transparent talk are indefensible. A third position accepts the critique and changes the question. If the account is made in the encounter, the encounter is the finding, and the problems become tractable once talk is studied as talk.

Argument will not settle this, because the sides answer different questions about what an interview is for. What could settle it is a criterion-validity test the two traditions already imply. Take a behaviour whose determinants the researcher has manipulated, and ask participants concurrently and then retrospectively about it. Compare each report with the manipulation. Nisbett and Wilson (1977) and Wilson and Dunn (2004) predict the retrospective explanation will substitute a plausible theory for the actual cause. Ericsson and Simon (1980) predict the concurrent report will track the heeded information. A study that ran that comparison on a realistic interface task would tell a designer which question to ask rather than which side to join. It would report where each kind of report matched and failed. No such benchmark exists in this unit's literature.

The two literatures are separated by scope rather than by findings. Nisbett and Wilson rest on experimental reviews of what people report about causes. Ericsson and Simon rest on a model of what a speaker heeds while working. Roulston, Kvale and Brinkmann, and Potter and Hepburn argue from the theory and practice of interviewing rather than from a trial. While the dispute is open, keep the report close to the task it describes. Ask about what the participant is doing rather than why they did it, and do not ask them to forecast what they will do. The strongest objection is that a concurrent report can be evidence about process rather than only about talk, where the speaker heeds the information as they work.

Where the argument is heading

The direction is towards treating the interview as one bounded instrument among several rather than a neutral default. Lo Iacono, Symonds and Brown (2016) already judge remote interviewing on its own terms rather than as a degraded shared room. Ericsson and Simon (1980) with Wilson and Dunn (2004) give the rule for which reports carry weight. Roulston (2010) and Denzin (2001) make the interviewer's reflexivity a condition of the claim. Nunkoosing (2005) with Atkinson and Silverman (1997) set the ceiling any such claim must respect. A methods section of the next few years should name the mechanism its reports rest on. It should say what the encounter did to the data, and pair the interview with observation wherever the question is about behaviour.

The sources this unit rests on

• Nisbett, R. E. and Wilson, T. D. (1977). Telling more than we can know: Verbal reports on mental processes. Psychological Review. doi:10.1037/0033-295x.84.3.231 Experimental review finding that people have little or no direct introspective access to higher-order cognitive processes, so reports of process draw on a priori causal theories rather than an internal record of what influenced the response. It is the founding result behind the unit's warning about asking for causes.
• Wilson, T. D. and Dunn, E. W. (2004). Self-Knowledge: Its Limits, Value, and Potential for Improvement. Annual Review of Psychology. doi:10.1146/annurev.psych.55.090902.141954 Review concluding that introspection cannot provide a direct pipeline to mental processes, while indirect routes such as observing one's own behaviour can improve self-knowledge. It states when self-report can and cannot be trusted.
• Ericsson, K. A. and Simon, H. A. (1980). Verbal reports as data. Psychological Review. doi:10.1037/0033-295x.87.3.215 Argues that verbal reports are data whose validity depends on the mechanisms generating them, modelling a participant thinking aloud as verbalising what they are heeding in short-term memory. It supplies the conditions under which concurrent talk is defensible.
• Kvale, S. and Brinkmann, S. (2009). InterViews: Learning the Craft of Qualitative Research Interviewing. 2nd edition. Thousand Oaks, CA: SAGE. url:https://archive.org/details/interviewslearni0000kval Treats the interview as a craft with stages, from thematising the question to analysing and reporting, and argues that quality is settled at design and cannot be recovered in analysis. It grounds the claim that a badly posed study cannot be repaired later.
• Gubrium, J. and Holstein, J. (2001). Handbook of Interview Research. SAGE Publications, Inc. doi:10.4135/9781412973588 Edited handbook surveying the interview from structured survey interviewing to life history and poststructural approaches, arguing that it is an active encounter in which both parties construct the account. It shows that interviewing is several incompatible traditions, not one method.
• Roulston, K. (2010). Reflective Interviewing: A Guide to Theory and Practice. SAGE Publications Ltd. doi:10.4135/9781446288009 Guide to design, conduct and analysis that sets competing theories of what an interview produces against each other and argues for reflexivity about the interviewer's role. It supports the claim that the researcher's perspective is part of the method.
• Rapley, T. J. (2001). The art(fulness) of open-ended interviewing: some considerations on analysing interviews. Qualitative Research. doi:10.1177/146879410100100303 Argues that analyses have attended to the co-construction of accounts but neglected the interviewer, and that the local interactional detail of the encounter should be the analyst's object. It moves analysis from what talk reports to how the account was produced.
• Potter, J. and Hepburn, A. (2005). Qualitative interviews in psychology: problems and possibilities. Qualitative Research in Psychology. doi:10.1191/1478088705qp045oa Distinguishes contingent problems, such as the deletion of the interviewer, from necessary problems such as the flooding of the encounter with social science agendas and categories. It argues these are tractable once interview talk is the phenomenon under study.
• Atkinson, P. and Silverman, D. (1997). Kundera's Immortality: The Interview Society and the Invention of the Self. Qualitative Inquiry. doi:10.1177/107780049700300304 Examines the rise of the interview as the central instrument of social research and its relation to the production of the self, arguing that an account is shaped by the genre and by strategies for presenting a life. It states the case that ubiquity is not validity.
• Nunkoosing, K. (2005). The Problems With Interviews. Qualitative Health Research. doi:10.1177/1049732304273903 Critically examines the interview through power and resistance, the distinction between truth and authenticity, and the difficulty of consent when understanding is uncertain for both parties. It is the sceptical position the unit's rules have to answer.
• Lo Iacono, V., Symonds, P. and Brown, D. H. K. (2016). Skype as a Tool for Qualitative Research Interviews. Sociological Research Online. doi:10.5153/sro.3952 Draws on studies of dance and wayfinding to weigh the reach and low cost of voice-over-internet interviewing against its effects on rapport, non-verbal cues and ethics. It concludes the medium is a viable complement to face-to-face rather than a replacement.
• Denzin, N. K. (2001). The reflexive interview and a performative social science. Qualitative Research. doi:10.1177/146879410100100102 Locates the reflexive interview within the cinematic-interview society and develops the performance interview and performative writing. It is the theoretical statement that reflexivity about the encounter is a requirement of validity.