Uncertainty for non-specialists: intervals, ranges, and what readers actually infer

An estimate drawn without its interval invites a reader to treat it as a fact. The remedy is to draw the interval. This fourth of ten units argues that the remedy is not neutral. The same statistic drawn as an error bar, a confidence interval, a prediction interval or an animated draw licenses different inferences from readers who understood them all. A display can still mislead about the direction of an effect, so the design problem is matching representation to required inference.

A designer must keep three ideas apart. Inferential uncertainty is uncertainty about an estimate, carried by standard errors and intervals. Outcome uncertainty is variation in individual outcomes around it, carried by standard deviations and prediction intervals. Reading uncertainty is what a reader infers when one of the two is drawn.

Uncertainty is many ideas, and an interval fixes only one of them

Uncertainty is not one object with one correct picture. Kale, Kay and Hullman (2019) interviewed eleven professional researchers who synthesise evidence for decisions and analysed 480 analytical decisions. They found a garden of forking paths, where each choice can shift the finding. Their uncertainty is about which analysis is right, not the width of one error bar. Hofman, Goldstein and Hullman (2020) add a second contrast. A result can be shown with inferential uncertainty, the precision of an estimate, or with outcome uncertainty, the variation of outcomes around it.

Olston and Mackinlay (2002) add a third kind that error bars misrepresent. A reading known to lie inside a range carries bounded uncertainty, which is not a probability distribution. An error bar asserts properties, such as a spread around a mean, that a bound does not have. Their remedy draws the bound in place of a standard deviation.

Readers do not read intervals the way the guidelines assume

Belia, Fidler, Williams and Cumming (2005) asked journal authors to adjust a figure until two means with error bars looked just significant. The authors came from psychology, behavioural neuroscience and medicine. Of 473 respondents, many leading researchers held severe misconceptions about error bars and significance. They did not distinguish confidence intervals from standard error bars.

Cumming and Finch (2005) supply the rule the respondents were guessing at.

Two independent 95 per cent confidence intervals support a difference near the conventional threshold when they overlap by about half a margin of error. Intervals that merely touch correspond to a p value near 0.01 (after Cumming and Finch, 2005; Cumming, Fidler and Vaux, 2007).

Cumming, Fidler and Vaux (2007) make the rule conditional on the design. With ten or more observations per group, overlap of half of one arm corresponds to the conventional threshold. With three per group, a full arm must overlap. The correct reading belongs to the sample size, not the picture.

The encoding changes the inference, and it can leave a reader more confident and more wrong

If readers misread a correct interval, the encoding itself decides the inference. Correll and Gleicher (2014) ran crowd-sourced experiments on mean-and-error displays. Bar charts with error bars carry a within-the-bar bias. Readers read points inside the bar as more likely than points outside it, and they predicted larger effects with more confidence than the data supported. Gradient and violin plots moved judgements closer to expectation. Sanyal, Zhang, Bhattacharya, Amburn and Moorhead (2009) tested four encodings across four tasks and found no overall winner. Error bars underperformed throughout.

Hofman, Goldstein and Hullman (2020) found that readers shown confidence intervals overestimated a treatment effect. They were willing to pay more for it than readers shown prediction intervals, and they understated outcome variation.

Animated draws let the reader count outcomes rather than decode a statistic

Hullman, Resnick and Adar (2015) studied hypothetical outcome plots, which animate finite draws from a distribution. For judgements about two and three quantities at once, readers using the animated draws were markedly more accurate than readers using error bars or violin plots. They were weaker at estimating a high-variance mean. The representation helped because a reader counted outcomes rather than decoded a statistic. A reader who cannot see the marks, such as a screen-reader user, is served by no error bar or animation, so the value and its bound belong in text beside the graphic.

Showing value and uncertainty together costs one visual channel

Correll, Moritz and Heer (2018) describe bivariate maps that encode value and uncertainty with two visual variables. They note that interference between the channels reduces how well marks can be told apart. Their value-suppressing uncertainty palettes budget the channel unevenly. Value gets a wide colour range where uncertainty is low, and a narrow range where it is high, so unreliable regions become hard to read. Readers weighted the uncertainty more heavily than with a conventional map.

At map scale the same problem meets the areal unit

Maps concentrate every hazard at once. MacEachren and colleagues (2005) reviewed how uncertainty is conceptualised, how it enters decision making, how frameworks represent it and how displays are evaluated. They named seven research challenges, most about the passage from display to decision. Kinkeldey, MacEachren and Schiewe (2014) reviewed the empirical side and found it thin. They identified 44 user studies across 34 publications, against a much larger literature that invents ways to depict uncertainty.

The risk communication literature puts the audience in charge

The risk-communication literature meets these questions with a real audience. Spiegelhalter (2017) reviewed the techniques used to communicate statistical risk. He concluded that the primary message is to acknowledge expert guidance, be clear about objectives, and work closely with the audience. No single representation suits every member of an audience, so a designer should use several formats. Graphics convey gist while numbers carry detail, so an interval alone asks readers to take detail from a picture built for gist.

The contested question: does better design improve decisions, or only change confidence?

The literature disagrees about what an uncertainty display is for, and what counts as working. One position holds that a well-chosen encoding aligns a reader's judgement with the statistical inference. It treats error bars as the problem, and gradient plots, violin plots, animated draws and value-suppressing palettes as the remedy.

A second position holds that an accurate display can still mislead. On this view the first camp's improvement is on an abstract task rather than the reader's decision. Hofman, Goldstein and Hullman (2020) found that readers given confidence intervals overestimated treatment effects. Hullman, Qiao, Correll, Kale and Kay (2019) found that evaluation practice leans on performance and satisfaction measures, which assume more predictable judgement than the research supports.

The evidence that would settle it is a decision-relevant experiment rather than a better accuracy score.

1. hold the statistic constant and vary only the representation.
2. give the reader a decision with a defined correct action, such as whether to fund a service.
3. measure whether the action improves and whether confidence is calibrated to it, not merely whether a judged value is closer to a target.

Hullman and colleagues (2019) supply the structure in their taxonomy of evaluation decisions. If redesigned encodings improve the action, the first camp is right. If they only raise confidence, the second is.

The two positions rest on different kinds of evidence. The first rests on controlled comparisons of encodings (Correll and Gleicher, 2014; Hullman, Resnick and Adar, 2015). The second rests on a controlled comparison of interval formats (Hofman, Goldstein and Hullman, 2020) and a survey of 86 user studies (Hullman and colleagues, 2019). While the dispute is open, a designer should choose the representation for the decision the reader has to make, and report the confidence it produced beside the accuracy it improved. The strongest objection is that both sides measure abstract judgements rather than decisions, so the claim that representation must match the required inference holds only for the tasks the studies set.

Where this is heading

The direction is away from a single correct interval and towards representations chosen for the inference and the decision-maker. Kinkeldey, MacEachren and Schiewe (2014) call for evaluation against what a display changes rather than how it reads. Hullman and colleagues (2019) propose a taxonomy so that evaluation stops assuming predictable judgement. Kale, Kay and Hullman (2019) and Spiegelhalter (2017) both end at the audience, recommending distributions of possible outcomes, frequency framing and several formats. The open question is whether a display's correctness can be defined by the decision it improves rather than by the statistic it draws.

The sources this unit rests on

Each note states what the cited work did and found and what it adds to the argument. Identifiers and metadata were checked on 2 November 2026.

• Belia, S., Fidler, F., Williams, J. and Cumming, G. (2005). Researchers Misunderstand Confidence Intervals and Standard Error Bars. Psychological Methods, 10(4), 389-396. doi:10.1037/1082-989x.10.4.389 Asked journal authors in psychology, behavioural neuroscience and medicine to adjust a figure until two means with error bars looked just significantly different. Many of the 473 respondents held severe misconceptions about error bars and did not distinguish confidence intervals from standard error bars.
• Correll, M. and Gleicher, M. (2014). Error Bars Considered Harmful: Exploring Alternate Encodings for Mean and Error. IEEE Transactions on Visualization and Computer Graphics, 20(12), 2142-2151. doi:10.1109/tvcg.2014.2346298 Ran crowd-sourced experiments on how mean and error are encoded. Bar charts with error bars carry a within-the-bar bias and lead readers to predict larger effects with more confidence than the data support, while gradient and violin plots brought judgements closer to expectation.
• Correll, M., Moritz, D. and Heer, J. (2018). Value-Suppressing Uncertainty Palettes. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, 1-11. doi:10.1145/3173574.3174216 Propose palettes that give value a wide range of a channel when uncertainty is low and a narrow range when it is high. Readers weighted uncertainty more heavily than with a conventional map.
• Cumming, G. and Finch, S. (2005). Inference by Eye: Confidence Intervals and How to Read Pictures of Data. American Psychologist, 60(2), 170-180. doi:10.1037/0003-066x.60.2.170 Set out how confidence intervals should be read from a figure, arguing that psychologists seldom use or understand them. Give the rule of eye in which two independent 95 per cent intervals overlap by about half a margin of error at the threshold.
• Cumming, G., Fidler, F. and Vaux, D. L. (2007). Error bars in experimental biology. The Journal of Cell Biology, 177(1), 7-11. doi:10.1083/jcb.200611141 Set out eight rules for interpreting error bars, conditional on sample size. With ten or more observations per group, overlap of half of one arm corresponds to the threshold.
• Hofman, J. M., Goldstein, D. G. and Hullman, J. (2020). How Visualizing Inferential Uncertainty Can Mislead Readers About Treatment Effects in Scientific Results. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 1-12. doi:10.1145/3313831.3376454 Compared 95 per cent confidence intervals with 95 per cent prediction intervals for the same results. Readers shown confidence intervals overestimated the treatment effect and its probability of superiority and understated outcome variation; axis rescaling helped less than prediction intervals or animated draws.
• Hullman, J., Resnick, P. and Adar, E. (2015). Hypothetical Outcome Plots Outperform Error Bars and Violin Plots for Inferences about Reliability of Variable Ordering. PLOS ONE, 10(11), e0142444. doi:10.1371/journal.pone.0142444 Introduced hypothetical outcome plots, which animate a finite set of draws from a distribution, and compared them with error bars and violin plots. Readers were markedly more accurate with the animated draws for two and three quantities at once, and weaker at estimating a high-variance mean.
• Hullman, J., Qiao, X., Correll, M., Kale, A. and Kay, M. (2019). In Pursuit of Error: A Survey of Uncertainty Visualization Evaluation. IEEE Transactions on Visualization and Computer Graphics, 25(1), 903-913. doi:10.1109/tvcg.2018.2864889 Built a taxonomy of the decisions an uncertainty evaluation must make and applied it to 86 user studies. Evaluation practice concentrates on performance and satisfaction measures that assume more predictable judgement than the research supports.
• Kale, A., Kay, M. and Hullman, J. (2019). Decision-Making Under Uncertainty in Research Synthesis. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, 1-12. doi:10.1145/3290605.3300432 Interviewed eleven researchers who synthesise evidence for decisions and analysed 480 analytical decisions. Described a garden of forking paths where each choice can shift a finding, and recommended visualisations of possible outcomes and a frequency framing.
• Kinkeldey, C., MacEachren, A. M. and Schiewe, J. (2014). How to Assess Visual Communication of Uncertainty? A Systematic Review of Geospatial Uncertainty Visualisation User Studies. The Cartographic Journal, 51(4), 372-386. doi:10.1179/1743277414y.0000000099 Systematically reviewed the empirical literature on uncertainty visualisation and found that only a small part evaluates its techniques, identifying 44 user studies across 34 publications, concentrated on how displays are read rather than what they change.
• MacEachren, A. M., Robinson, A., Hopper, S., Gardner, S., Murray, R., Gahegan, M. and Hetzler, E. (2005). Visualizing Geospatial Information Uncertainty: What We Know and What We Need to Know. Cartography and Geographic Information Science, 32(3), 139-160. doi:10.1559/1523040054738936 Reviewed how uncertainty is conceptualised, how it enters decision making, how frameworks represent it and how displays are evaluated, then named seven research challenges. Argues that uncertainty is an intricate idea with many perspectives and no single representation.
• Olston, C. and Mackinlay, J. D. (2002). Visualising data with bounded uncertainty (the published title is American). IEEE Symposium on Information Visualization, 2002. INFOVIS 2002. doi:10.1109/infvis.2002.1173145 Argues that bounded uncertainty, such as a value known to lie within a range, differs in kind from statistical uncertainty, so error bars misrepresent it, and describes a technique for drawing the bound on common charts.
• Sanyal, J., Zhang, S., Bhattacharya, G., Amburn, P. and Moorhead, R. J. (2009). A User Study to Compare Four Uncertainty Visualization Methods for 1D and 2D Datasets. IEEE Transactions on Visualization and Computer Graphics, 15(6), 1209-1218. doi:10.1109/tvcg.2009.114 Had 27 users perform search and counting tasks on four uncertainty encodings: error bars, scaled glyph size, colour-mapped glyphs and colour-mapped surfaces. Found a significant interaction between encoding and task, with error bars underperforming.
• Spiegelhalter, D. (2017). Risk and Uncertainty Communication. Annual Review of Statistics and Its Application, 4, 31-60. doi:10.1146/annurev-statistics-010814-020148 Reviewed the techniques used to communicate statistical risk across health, weather, climate and other cases. Concludes that the message is to acknowledge expert guidance, be clear about objectives and work with the audience, and recommends using multiple formats.