Mixed methods design and the integration problem

Year 4 / Module 26 Advanced qualitative and mixed-methods research / Unit 7 / Mixed methods design and the integration problem

HCI4102 · Unit 7 of 10

Combining strands is not a method: a mixed methods design is defined by a named point of integration, and without one the honest label is two studies in one report.

A mixed label is earned where the strands meet

This seventh of ten units argues that a mixed methods study earns the label at the point where the strands meet, a point a design can name in advance and a reader can inspect afterwards. Morse (1991) supplied the notation for that obligation, separating studies that run the two approaches at once from those that run them in sequence, and marking which approach leads. The notation is a claim about location, not about quality. The six units before it read one kind of material.

Content is not a design, and the fit of integration decides the combination

A mixed methods study is usually described by what it contains: a survey plus interviews, a trial plus focus groups. Content is not a design. Fetters, Curry and Creswell (2013) set out integration at three levels. At the level of design stand three basic configurations, exploratory sequential, explanatory sequential and convergent, alongside four advanced frameworks. At the level of method the strands connect, build, merge or embed. At the level of interpretation and reporting they meet through narrative, data transformation or a joint display. The fit of integration names the property that decides whether the combination worked: the extent to which the two sets of findings cohere.

Where no such point can be named, the honest description is two studies in one report. That is not a verdict on either study; it states what the reader is asked to infer, since the combination is supposed to yield an inference that neither strand produces alone (Guetterman, Fetters and Creswell, 2015).

The field's own definitions centre on combination

Johnson, Onwuegbuzie and Turner (2007) asked leaders in mixed methods research for definitions and collected 19, most of which describe combination. They offered a summary definition in which research combines elements of qualitative and quantitative approaches "for the broad purposes of breadth and depth of understanding and corroboration". A questionnaire stapled to an interview schedule satisfies it.

Plano Clark and Ivankova (2016) define integration as an explicit interrelating of the two components, and place it at the centre of the approach. Fetters and Molina-Azorin (2017) note as journal editors that there is no widely agreed definition of integration at all. The field holds a permissive definition of the study and an aspirational definition of what makes it worth doing.

Five reasons to combine, one of which expects disagreement

Greene, Caracelli and Graham (1989) refined a framework against 57 empirical mixed method evaluations and named five purposes: triangulation, complementarity, development, initiation and expansion. Each implies a different design, and the analysis carries a warning. The term triangulation was commonly misused in evaluations that stated the purpose without a design able to deliver it. Relatively few evaluations integrated the method types at the level of data analysis.

One purpose runs against the habit of seeking agreement. Initiation deliberately seeks divergence, using the strands to surface paradox rather than to confirm one another. A study cannot adopt it and treat an inter-method discrepancy as a defect.

The audit record: declared more often than performed

O'Cathain, Murphy and Nicholl (2007) examined the proposals and reports of 75 mixed methods studies funded by an English commissioner of health services research between 1994 and 2004. They interviewed 20 researchers who worked on them. Mixed methods work was common at 18 per cent of 647 studies. The drivers were pragmatic, including strategic gains such as funding. The repertoire was narrow, mostly surveys and individual interviews.

Zhang and Creswell (2013) asked what the procedures were. They reviewed 30 health services research articles published between 2006 and 2010 and described three ways of mixing. The first analyses the two data types separately and integrates the results during interpretation. The second connects the strands in phases, so one builds on the other's findings. The third embeds one analysis within the other. Their conclusion restates the thesis: mixing is more than the combination of two independent components. Alone among the three, the first leaves both analyses exactly where they stood.

"Without integration, the knowledge yield is equivalent to that from a qualitative study and a quantitative study undertaken independently, rather than achieving a 'whole greater than the sum of the parts'." (O'Cathain, Murphy and Nicholl, 2010)

Integration is a third analytic task that begins after both analyses are finished and competes for time.

Integration has to be timetabled, because it is a third analytic task

O'Cathain, Murphy and Nicholl (2010) answer the practical objection with three techniques. A triangulation protocol lists the findings from each component on one page and asks where they agree, complement each other, fall silent or contradict. Silence counts as a result. A mixed methods matrix sets cases in rows and the data on each case in columns, so surprises within a case are examined before patterns across cases. Following a thread traces a theme from one component through the others.

Sampson, O'Cathain and Goodacre (2010) surveyed 682 patients and 486 carers across eight English hospitals, receiving 595 and 418 responses, and interviewed 33 patients and carers, 17 of them again after dissatisfaction with aftercare. The survey found that carers of patients on the new service rated the convenience of visiting as poorer than carers of patients on usual care. A questionnaire also decides who answers: the carer who reads English and has time to spare, rather than the one who does not. Held as three separate findings that is a result about convenience. Read beside the interviews, where the same carers gave the concern little weight because their relative's life had been saved, it becomes a result about what they were trading off.

Merging during analysis changes what the study finds

Moseholm and Fetters (2017) make the point analytic. For convergent studies they distinguish relational, methodological and directional dimensions of merging, and describe five named frameworks, including explanatory unidirectional and simultaneous bidirectional merging. Integration during analysis, they argue, changes what the study finds.

Guetterman, Fetters and Creswell (2015) supply the artefacts that make the claim checkable. Searching three journals that publish mixed methods research, they analysed 19 joint displays, mostly statistics-by-themes and side-by-side comparisons across four designs. Their exemplars carry an inference that neither strand produces alone. Bazeley (2018) gives the longest treatment of the analytic work, from planning for analysis through linked data, data transformation, dissonance and divergence, and warranted assertions.

Contested question: is integration a decision or a property of the study?

On the design position, integration is something a team chooses and can be held to. Fetters, Curry and Creswell (2013) supply the design vocabulary and the fit of integration, and O'Cathain, Murphy and Nicholl (2010) supply three techniques. Moseholm and Fetters (2017) supply analytic frameworks for convergent studies. Guetterman, Fetters and Creswell (2015) supply the joint display as an inspectable artefact. Creswell and Plano Clark (2018) supply the choice among seven designs. A claim of integration is then an empirical claim: a design that cannot name its point of contact has not integrated.

On the conceptual position, integration is a property of the whole study. Bazeley (2024) argues that it is constituted by interdependence that is connected, transactional, transformative, coherent and dynamic. Such a property is not created at one interface, because the strands change each other as the study proceeds. Fetters and Molina-Azorin (2017) support the position with a trilogy in which integration has philosophical, methodological and methods dimensions. If no definition is agreed, an audit for a named point audits one author's definition.

The disagreement cannot be settled by definition, and the evidence does not exist. It would need studies matched on question and design, each with a declared point of integration, and two tests. The first is a reader test: reviewers who do not know the declared design judge whether the reported inference depends on both strands. The second is a subtraction test: remove one strand and ask whether the other still supports the inference. Declared points that survive both would show the design position sufficient. Dynamic integration that fails the point audit would show the conceptual position right. The nearest existing evidence points the other way. Greene, Caracelli and Graham (1989) found declared purposes that no design could deliver, and Zhang and Creswell (2013) described procedures that leave each analysis where it stood.

The design position is evidenced; the conceptual position is not. O'Cathain, Murphy and Nicholl (2007) audited 75 funded studies and interviewed 20 researchers; Greene, Caracelli and Graham (1989) refined a framework against 57 evaluations. Both audit practice rather than test a model. Bazeley (2024) argues a five-part conceptual model from the literature and offers no study that tests it. While the dispute is open, a designer should declare the point of integration in advance, put the joint display in the report, and run a subtraction test on the main inference. The strongest objection is that a subtraction test measures the report rather than the study, since an undeclared interdependence can survive removal of a strand while the account cannot show it.

Where the argument is heading

The direction of travel is from technique towards reportable interdependence. Bazeley (2024) has moved the discussion from a list of procedures to a conceptual model of what integration is made of. Fetters and Molina-Azorin (2017) have widened it from methods to dimensions. The question the field is converging on is a reader's question: can someone who did not collect the data tell, from the report, whether the study integrated or merely contained?

The sources this unit rests on

Each note states what the cited document did and found. Records were checked against Crossref and PubMed, 1 November 2026.

Fetters, M. D., Curry, L. A. and Creswell, J. W. (2013). Achieving integration in mixed methods designs: principles and practice. doi:10.1111/1475-6773.12117 Names integration at design, method and interpretation levels, with three basic designs and four advanced frameworks. The fit of integration measures coherence.
Morse, J. M. (1991). Approaches to qualitative-quantitative methodological triangulation. doi:10.1097/00006199-199103000-00014 Separates simultaneous from sequential triangulation and introduces the notation for which approach leads and whether the strands run together or in sequence. The indexed record carries no abstract, so the note rests on citation.
Greene, J. C., Caracelli, V. J. and Graham, W. F. (1989). Toward a conceptual framework for mixed-method evaluation designs. doi:10.3102/01623737011003255 Refined a framework against 57 evaluations and named five purposes: triangulation, complementarity, development, initiation and expansion. It found triangulation commonly misused where the design could not deliver its purpose, and few studies integrating during analysis.
Johnson, R. B., Onwuegbuzie, A. J. and Turner, L. A. (2007). Toward a definition of mixed methods research. doi:10.1177/1558689806298224 Collected 19 definitions from leaders in the field and offered a summary definition built on combining qualitative and quantitative elements. The field's own definition centres on combination.
Plano Clark, V. L. and Ivankova, N. V. (2016). Mixed methods research: a guide to the field. doi:10.4135/9781483398341 A guide that defines integration as an explicit interrelating of the qualitative and quantitative components It supplies the unit's stricter definition.
Fetters, M. D. and Molina-Azorin, J. F. (2017). The Journal of Mixed Methods Research starts a new decade: the mixed methods research integration trilogy and its dimensions. doi:10.1177/1558689817714066 Proposes an integration trilogy spanning philosophical, methodological and methods dimensions, records that no definition is widely agreed.
O'Cathain, A., Murphy, E. and Nicholl, J. (2007). Why, and how, mixed methods research is undertaken in health services research in England: a mixed methods study. doi:10.1186/1472-6963-7-85 Analysed proposals and reports of 75 mixed studies and interviewed 20 researchers, having classed 18 per cent of 647 studies as mixed. Drivers were pragmatic, including funding, and the repertoire narrow.
Zhang, W. and Creswell, J. (2013). The use of "mixing" procedure of mixed methods in health services research. doi:10.1097/mlr.0b013e31824642fd Reviewed 30 articles and described three mixing procedures, the first integrating only during interpretation. Its conclusion is that mixing exceeds combining independent components.
O'Cathain, A., Murphy, E. and Nicholl, J. (2010). Three techniques for integrating data in mixed methods studies. doi:10.1136/bmj.c4587 Sets out three techniques: the triangulation protocol with its convergence coding matrix, following a thread, and the mixed methods matrix for linked cases. Without integration the yield is two independent studies.
Morgan, D. L. (2007). Paradigms lost and pragmatism regained. Journal of Mixed Methods Research, 1(1), 48-76. doi:10.1177/2345678906292462 Argues that the paradigm debate should give way to a pragmatic approach connecting philosophical concerns about knowledge with the methods used to produce it, It is the source the post-analysis effort the unit timetables is attributed to.
Sampson, F. C., O'Cathain, A. and Goodacre, S. (2010). Is primary angioplasty an acceptable alternative to thrombolysis? Quantitative and qualitative study of patient and carer satisfaction. Health Expectations, 13(4), 379-390. doi:10.1111/j.1369-7625.2009.00589.x The primary study behind the angioplasty case: a survey across eight English hospitals with 595 patient and 418 carer responses, plus interviews. It found visiting convenience rated poorer by carers at angioplasty sites.
Moseholm, E. and Fetters, M. D. (2017). Conceptual models to guide integration during analysis in convergent mixed methods studies. doi:10.1177/2059799117703118 Introduces a typology for merging data in convergent studies across relational, methodological and directional dimensions, with five named frameworks. It shows integration can be specified during analysis.
Guetterman, T. C., Fetters, M. D. and Creswell, J. W. (2015). Integrating quantitative and qualitative results in health science mixed methods research through joint displays. doi:10.1370/afm.1865 Analysed 19 joint displays, finding statistics-by-themes and side-by-side comparisons most prevalent. Its exemplars convey an inference neither strand produces alone.
Creswell, J. W. and Plano Clark, V. L. (2018). Designing and conducting mixed methods research (3rd ed.). Thousand Oaks, CA: Sage. https://searchworks.stanford.edu/view/12808920 The field's textbook, whose third edition presents seven designs and covers the sequence and weighting of strands. The plan's stable URL points to an earlier edition record, so the edition was verified against library catalogues.
Bazeley, P. (2018). Integrating analyses in mixed methods research. doi:10.4135/9781526417190 A book-length account of the analytic work, from planning through data transformation and divergence to warranted assertions. It treats integration as planned and conducted, not added at the writing stage.
Bazeley, P. (2024). Conceptualizing integration in mixed methods research. doi:10.1177/15586898241253636 Develops a conceptual model in which integration is constituted by interdependence that is connected, transactional, transformative, coherent and dynamic. It makes integration a property of the whole.

Check your understanding & apply it · 4 questions, 1 essay

1A paper describes its work as mixed methods because it administered a questionnaire and then interviewed some respondents, mentioning integration in the discussion. Which diagnosis does the unit support?

2A team uses Zhang and Creswell's (2013) first mixing procedure, analysing the two data types separately and integrating only at interpretation. What does the unit predict about the yield?

3Two reviewers disagree about whether a study integrated. One asks whether the report names a point of contact and shows a joint display. The other asks whether the strands changed each other over the life of the study. Which mapping does the unit support?

4A journal editor reads a submission whose two strands agree throughout and whose authors call the agreement triangulation. Applying Greene, Caracelli and Graham (1989), which judgement does the unit support?

5Essay A team runs a randomised trial of a clinical decision-support tool alongside interviews with clinicians, and drafts a paper that describes the work as mixed methods and mentions integration in the discussion. Advise them. Explain what their design choices have committed them to, state what the field's vocabulary requires them to name, report what the audit evidence says about studies that look like theirs, set out the techniques available if they want to integrate during analysis rather than during writing, state what the unit's contested question would demand of each side in their case, and say what evidence would settle it. Name your sources at each step.

The team's first task is to say what kind of design they have and where its strands meet, because content is not a design. Fetters, Curry and Creswell (2013) distinguish integration at the level of design, where the three basic configurations are exploratory sequential, explanatory sequential and convergent, at the level of method, where strands connect, build, merge or embed, and at the level of interpretation and reporting, where they meet through narrative, data transformation or a joint display; a trial with interviews is usually explanatory sequential or convergent, and the team must pick one and defend it. Morse (1991) shows that location is part of the decision, since her notation records whether the strands run together or in sequence and which one leads. The title they propose also has to survive the field's own looseness: Johnson, Onwuegbuzie and Turner (2007) collected 19 definitions from leaders, most of which turn on combination alone, so a claim of mixed methods is easy to make and proves little; Plano Clark and Ivankova (2016) define integration as an explicit interrelating of the components, which is the stronger bar. The audit evidence is discouraging. O'Cathain, Murphy and Nicholl (2007) classified 18 per cent of 647 health services research studies as mixed methods and found the drivers pragmatic, including funding, and Zhang and Creswell (2013) reviewed 30 articles and found three mixing procedures, the first of which analysed the two data types separately and integrated them only at interpretation. O'Cathain, Murphy and Nicholl (2010) supply the remedy: a triangulation protocol that lists findings on one page and counts silence and dissonance as results, a mixed methods matrix that sets cases in rows against the data held on each case, and following a thread, with the post-analysis work timetabled as Morgan's third effort. For a convergent design, Moseholm and Fetters (2017) offer named analytic frameworks that merge data during analysis, and Guetterman, Fetters and Creswell (2015) show in 19 published joint displays what an integrated result looks like when the arrangement carries an inference neither strand produces alone. The contested question then applies to them directly. The design position, held by Fetters, Curry and Creswell (2013) and O'Cathain and colleagues, demands that they name the point of integration and show it in a display; the conceptual position of Bazeley (2024), who argues that integration is constituted by interdependence that is connected, transactional, transformative, coherent and dynamic, and of Fetters and Molina-Azorin (2017), who spread integration across philosophical, methodological and methods dimensions, says that no single display captures what their two strands do to each other over the life of the study. What would settle it is a reader test, in which reviewers who do not know the declared design judge whether the reported inference depends on both strands, and a subtraction test, in which one strand is removed and the inference is re-examined. Greene, Caracelli and Graham (1989) already showed why the tests matter: they found evaluations that declared the purpose of triangulation without a design able to deliver it, which is the failure the team can avoid by naming the join before they write.