Content analysis and the discipline of counting

Content analysis began as a method for the objective, systematic and quantitative description of the manifest content of communication, and over time it expanded to include latent content (Graneheim and Lundman, 2004). Counting buys reach, reproducibility and comparability: thousands of messages can be classified, a rule-guided procedure repeated on another corpus, and two periods or platforms placed on one scale. None of these gains is meaning. This unit argues that the step from frequency to significance must be made visible enough to be contested. A report cannot present the count as though it had settled the question.

The thesis is not that counting is a poor method. Hsieh and Shannon (2005) describe the two parts in what they call the summative approach: the analyst counts keywords or content, then interprets the underlying context. When that interpretation is written as an aside, the reader receives a number as a finding, and cannot disagree with the reasoning that turned it into one. Counting is therefore a reporting discipline: say which count was taken, what it cannot distinguish, and why it supports that reading. The two units before it described families that interpret. This sixth of ten units takes the family that counts first.

The counting tradition names its standard, and classification is the analytic step

Weber (1990) gives the compact statement of the position. His short text moves from content classification and interpretation, through the techniques of content analysis, to issues grouped under measurement, indication, representation and interpretation. Classification is the analytic step: a category system is not a filing cabinet that leaves the material untouched, but a decision about which differences in the text can be seen.

Neuendorf (2017) presents content analysis as a systematic, quantitative and largely deductive method. She gives reliability a chapter covering chance-corrected agreement, unitising reliability, the reliability subsample, and manifest as against latent content. If two coders cannot agree where a unit begins, the count is not measuring what it claims. Kondracki, Wellman and Amundson (2002) show the procedure as a design decision: the method chosen for nutrition education depends on the material, the results and the researcher's capability.

One family offers three approaches, and only one of them counts first

Hsieh and Shannon (2005) separate conventional, directed and summative approaches. The authors claim that all three interpret meaning from text and adhere to the naturalistic paradigm. Only the summative approach begins with a count, so a demand for counting is a demand for one member of a family. That paradigm claim is where the literature disagrees. Neuendorf's (2017) deductive method belongs to another account of what makes content analysis good. Mayring (2000) tries to hold both together with a rule-guided qualitative text analysis that preserves some strengths of the quantitative kind.

Rules that make the procedure repeatable

Elo and Kyngas (2008) give the procedural backbone. They set out inductive and deductive content analysis as three phases: preparation, organising and reporting. They add a decision rule: use the inductive route where the phenomenon has no prior studies, the deductive route where a previous theory is tested. The rule makes the analyst's reading of the literature part of the reported method. Bengtsson (2016) pushes the same logic earlier, arguing that credibility begins when the study is planned, before any coding. He sets out four stages: decontextualisation, recontextualisation, categorisation and compilation.

The step from frequency to significance has to be shown, not assumed

Graneheim, Lindgren and Lundman (2017) name the central difficulty: content analysis has changed from a counting game to a more interpretative approach. The key issue is to distinguish the level of abstraction from the degree of interpretation, since increased abstraction raises the burden of demonstrating credibility. Their demand is to show the logic by which categories and themes were abstracted and connected to the aim. Lindgren, Lundman and Graneheim (2020) extend it to where meaning units are selected and coded. The analysis can be descriptive and interpretative at once, reaching depth when the data allow latent interpretation. Kleinheksel and colleagues (2020) describe the manifest and latent approaches with worked examples, and set out threats to reliability and validity with mitigations. The count itself can now be made without a human coder: Bijker and colleagues (2024) had ChatGPT detect change mechanisms in 537 forum posts and build the coding schemes from them, with only 66 to 88 per cent of its identified mechanisms matching human coding, and the authors did not assess validity.

What a category cannot carry

A count is only as good as its unit, and the unit is a judgement. Erlingsson and Brysiewicz (2017) work from the data to meaning units, codes and categories for novice researchers, making the decisions visible. In their case, a patient's "They pushed me into the middle of the room and then walked away... they just left me" is condensed to "left alone", given the code "left alone", and placed in the category "staff actions and non-actions". A reader can disagree at any of those steps, which is what makes the judgement inspectable. A team codes ten thousand forum posts for requests for help and reports that mentions of cost rose after a price change. The count is sound and reproducible, but cannot distinguish a new complaint from a returning one with a new vocabulary, because the code marks a topic rather than the relation between speaker and system, or the second language the post was written in. The repair is not a better number but a sentence stating what the code cannot separate, and a reading of the posts it could not distinguish.

Contested question: is the count the finding, or the input to one?

The first position is that a properly produced count is a finding in its own right, on prescriptive evidence. Neuendorf (2017) systematises the tradition in a textbook, Weber (1990) introduces it in a short text, and Downe-Wamboldt (1992) supplies the justifying aim: external validity, which unlike strictly qualitative designs the method can claim. On this view quality is secured before interpretation, by a defensible sampling frame, a specified instrument and an adequate reliability figure, and the skill lies in choosing the count well.

The second position is that the count is an input and the finding is the interpretation. Its evidence is thinner and of the same kind. Hsieh and Shannon (2005) state it in a methods article whose illustrations are hypothetical end-of-life cases rather than results from studies, which is itself informative. Graneheim, Lindgren and Lundman (2017) and Lindgren, Lundman and Graneheim (2020) argue it in discussion papers, and Kleinheksel and colleagues (2020) in a teaching article. Mayring (2000) occupies the middle ground by design. Neither side offers a systematic comparison of published reports, so the dispute is not yet an empirical one.

What would settle it is the audit neither side has run: take content analyses that report both a distribution and an interpretation, and code each for whether the interpretation can be contested separately from the count, and whether reliability covered the coding of units or also the interpretive move. The first predicts that the interpretive step will not appear in the findings, because it was never part of the instrument. The second predicts that it will appear in the discussion as an unsupported addition.

The strongest counter-argument to this unit's own claim is that a specified instrument with chance-corrected reliability is where validity lives. On that reading the count is the transferable finding, the reading is commentary, and an interpretive gloss for every frequency multiplies prose no reader can check.

What to do while the dispute is open

Report the count with the unit rule and with what the code cannot separate. Give the reliability figure for the coding of units, and mark the interpretive move as a separate claim with its own warrant; where a count drives a decision, name the alternative reading it cannot exclude.

Where the argument is heading

The direction of travel is towards reporting that separates the count from the interpretation and defends both. Vears and Gillam (2022) show the demand has not faded: beginners meet competing terminologies, and the choice of variant changes the claim a study can make. Lindgren and colleagues (2020) push the argument to each point where the analyst acts. Graneheim and colleagues (2017) supply the criterion: the demonstration of the logic connecting categories to the aim. The risk is that a study which compresses its interpretive step is judged by the half it kept, and the deeper problem goes unrepaired.

The sources this unit rests on

• Bijker, R., Merkouris, S. S., Dowling, N. A. and Rodda, S. N. (2024). Bijker and colleagues had ChatGPT detect change mechanisms in 537 forum posts and build ten coding schemes, with only 66 to 88 per cent of its identified mechanisms matching human coding. The count can now be made without a human coder, and the authors assessed reliability, not validity.
• Bengtsson, M. (2016). How to plan and perform a qualitative study using content analysis. NursingPlus Open 2, 8-14. doi:10.1016/j.npls.2016.01.001 Bengtsson argues that credibility begins when the study is planned, before any coding, and sets out four stages, decontextualisation, recontextualisation, categorisation and compilation. It shows the interpretive decisions distributed across the study, not confined to a final step.
• Downe-Wamboldt, B. (1992). Content analysis: Method, applications, and issues. Health Care for Women International 13(3), 313-321. doi:10.1080/07399339209516006 Downe-Wamboldt sets out the methodology and argues that, unlike strictly qualitative designs, the method has external validity as its goal. It states what counting is for in the tradition that defends it.
• Elo, S. and Kyngas, H. (2008). The qualitative content analysis process. Journal of Advanced Nursing 62(1), 107-115. doi:10.1111/j.1365-2648.2007.04569.x Elo and Kyngas describe inductive and deductive content analysis as three phases, preparation, organising and reporting, and give a decision rule, inductive where no prior studies exist, deductive where a previous theory is tested. It provides the unit's procedural backbone.
• Erlingsson, C. and Brysiewicz, P. (2017). A hands-on guide to doing content analysis. African Journal of Emergency Medicine 7(3), 93-99. doi:10.1016/j.afjem.2017.08.001 Erlingsson and Brysiewicz give a practical example for novice researchers, working from the data through meaning units and codes to categories. It shows that the unit of analysis is a judgement made in the open rather than a property of the text.
• Graneheim, U. H. and Lundman, B. (2004). Qualitative content analysis in nursing research: Concepts, procedures and measures to achieve trustworthiness. Nurse Education Today 24(2), 105-112. doi:10.1016/j.nedt.2003.10.001 Graneheim and Lundman record that the literature shows conflicting opinions and unsolved issues about the concepts, procedures and interpretation of the method, and define manifest and latent content, meaning unit, condensation and category. They propose measures for credibility, dependability and transferability, which shows the disagreement to be internal.
• Graneheim, U. H., Lindgren, B.-M. and Lundman, B. (2017). Methodological challenges in qualitative content analysis: A discussion paper. Nurse Education Today 56, 29-34. doi:10.1016/j.nedt.2017.06.002 Graneheim, Lindgren and Lundman observe that content analysis has changed from a counting game to a more interpretative approach, and argue that the key issue is to distinguish the level of abstraction from the degree of interpretation. They require the analysis to show the logic connecting categories to the aim.
• Hsieh, H.-F. and Shannon, S. E. (2005). Three approaches to qualitative content analysis. Qualitative Health Research 15(9), 1277-1288. doi:10.1177/1049732305276687 Hsieh and Shannon separate conventional, directed and summative approaches, differing in their coding schemes, the origins of their codes and their threats to trustworthiness, and argue that all three adhere to the naturalistic paradigm. In the summative approach counting is followed by an interpretation of the context, which is the unit's two-step structure.
• Kleinheksel, A. J., Rockich-Winston, N., Tawfik, H. and Wyatt, T. R. (2020). Demystifying content analysis. American Journal of Pharmaceutical Education 84(1), 127-137. doi:10.5688/ajpe7113 Kleinheksel and colleagues describe manifest and latent approaches with worked examples and set out threats to reliability and validity with mitigations. It shows the two-part analysis taught as method, not conceded as a limitation.
• Kondracki, N. L., Wellman, N. S. and Amundson, D. R. (2002). Content analysis: Review of methods and their applications in nutrition education. J. Nutr. Educ. Behav. 34(4), 224-230. doi:10.1016/s1499-4046(06)60097-3 Kondracki, Wellman and Amundson review content analysis as a set of qualitative and quantitative methods and argue that the method chosen depends on the material, the results and the researcher's capability. That makes the procedure a design decision, not a fixed recipe.
• Lindgren, B.-M., Lundman, B. and Graneheim, U. H. (2020). Abstraction and interpretation during the qualitative content analysis process. International Journal of Nursing Studies 108, article 103632. doi:10.1016/j.ijnurstu.2020.103632 Lindgren, Lundman and Graneheim answer the charge that standardised methods amount to superficial sorting, examining how meaning units are selected and coded. They argue that the analysis can be descriptive and interpretative at once, reaching depth when the data allow latent interpretation, which makes the interpretive step internal.
• Mayring, P. (2000). Qualitative content analysis. Forum: Qualitative Social Research 1(2), article 20. doi:10.17169/fqs-1.2.1089 Mayring sets out a systematic, rule-guided qualitative text analysis preserving the strengths of quantitative content analysis, working with units of analysis, step models, categories, validity and reliability. Its procedures, the inductive development and the deductive application of categories, hold the counting and interpretive traditions together.
• Neuendorf, K. A. (2017). The Content Analysis Guidebook. 2nd edition. Thousand Oaks: Sage. doi:10.4135/9781071873045 Neuendorf presents content analysis as a systematic, quantitative and largely deductive method, and gives reliability a chapter running through chance-corrected agreement, unitising reliability, the reliability subsample, and manifest as against latent content. The list is the tradition's answer to meaning: convert it into measurement, so a sound count can stand as the finding.
• Vears, D. F. and Gillam, L. (2022). Inductive content analysis: A guide for beginning qualitative researchers. Focus on Health Professional Education 23(1), 111-127. doi:10.11157/fohpe.v23i1.544 Vears and Gillam guide beginning researchers through inductive content analysis, suited to small-scale research, and observe that the literature is hard to navigate because its terminology varies. They distinguish it from deductive content analysis and from thematic analysis, which makes the variant a live choice.
• Weber, R. P. (1990). Basic Content Analysis. 2nd edition. Newbury Park: Sage. doi:10.4135/9781412983488 Weber's short text moves from content classification and interpretation to the techniques of content analysis and to issues grouped under measurement, indication, representation and interpretation. The arrangement treats classification as the analytic step. It gives the compact argument that the instrument, not the text, sets the terms of the count.