Perception and the ranking of visual encodings

Data visualisation is often taught from one empirical object: an ordering of visual encodings by the accuracy with which readers judge values. Position on a common scale sits at the top, and area, angle and colour saturation at the bottom. The design advice drawn from it is often presented as if the evidence licensed it. This first of ten units argues that the ordering is a robust ordinal result for isolated abstract judgements and under-determines design, and the nine that follow each take one limit of it. A real display uses several channels at once, and a ranking measured on separated variables does not predict their combination. A reader of a real display is also answering a task, not estimating a magnitude on its own, whether that task is a change over time or one total against another. The ranking tells you which way a chart will tend to fail; it does not tell you what to draw.

The theory came before the measurement

Bertin (2011) set out the account the ranking would later be measured against, treating a graphic as a system of signs in which a small set of visual variables is matched to the properties of the data they carry, and a variable may encode data only if it has the corresponding perceptual property.

Cleveland and McGill (1984) measured it. Readers judged the relative size of two encoded values in abstract displays. The pattern of errors then ordered a set of elementary perceptual tasks. Position on a common scale and length were the most accurate, and area, angle and colour saturation the least. The authors drew one guideline, to use tasks high in the ordering, and reported that the task set should be expanded.

Mackinlay (1986) turned the ordering into a machine rule. He defined a language for graphical presentations with expressiveness and effectiveness criteria, and built a system choosing an encoding from a ranked list of perceptual codes. Effectiveness was an ordering of visual variables, so the automatic designer inherited the ranking rather than measuring reader performance.

What the ranking is, and what it is not

The ranking's strongest support is a replication. Heer and Bostock (2010) reran the judgement experiments with crowdworkers and used the same error measure. The shape and order of the judgement types were preserved, position still significantly outperformed length, and length judgements improved on the original. The gaps between channels narrowed, but what survives is an order.

They also tested rectangular area. Accuracy matched circular area, but pairs with an aspect ratio of one, the squares a squarified treemap aims for, produced the worst judgements.

The paper states its own limits, and they are the limits of this unit. Display, lighting and viewing distance were uncontrolled, and only a subclass of perception experiments can run this way. Visual variables also interact, so extrapolating from isolated variables is unreliable.

Cleveland and McGill (1986) ran a further experiment on six basic judgements and recovered the same order. The two position judgements were the most accurate, length came second, angle and slope third, and area last. Accuracy fell as the distance between the judged objects increased. The level of technical training of the subject made no difference. The arrangement of a display therefore moves a reading that the ranking treats as a fixed property of the channel.

The judgement decides the chart, and the doubt is older than the consensus

Simkin and Hastie (1987) analysed graph reading as a sequence of component operations. They found that display type interacted with task, so no single form was best. They argued that design should follow the component processes a reader must execute.

Hollands and Spence (1992) tested that interaction directly. In two experiments they compared how accurately readers judged change and proportion from line, bar and pie forms. They found the two judgements drew on different perceptual strategies.

Spence and Lewandowsky (1991) reached the same place from proportions. They found that the form made little difference for some proportion judgements, while errors were large for others. They concluded that the display should be matched to the particular judgement required. A blanket ban on pie charts is therefore wrong as stated and right as a warning.

Quadri and colleagues (2024) took the high-level case. Readers describing line graphs, bar graphs and scatterplots comprehended patterns that the statistics of classical perception studies did not predict, and chart type alone did not decide what they extracted.

The ranking did not arrive uncontested and then acquire critics. Lewandowsky and Spence (1989) reviewed the psychological evidence on graph reading. They reported reliable differences among encodings for particular judgements, and found the evidence base thinner than the field assumed.

A display carries more than accuracy

A ranking on judgement accuracy assumes accuracy is the property a display is for. Borkin and colleagues (2013) tested another, showing crowdworkers hundreds of visualisations in a memory test. They found memorability a stable property of the image, only weakly related to chart type or aesthetic rating, while colour, density and recognisable elements predicted recall.

Zacks and Tversky (1999) showed that the form changes what a reader takes the data to be about. They asked participants to describe and draw bar and line graphs and found readers took lines as trends over time and bars as discrete comparisons.

Shah and Freedman (2011) modelled comprehension as an interaction of bottom-up perceptual encoding with top-down knowledge and expectation. They found prior beliefs and graph form jointly determined what readers took from a display. That places the ranking on the bottom-up half of the process, and it is silent on what the reader already believes and on the reader who needs a screen reader.

Contested question: is the ranking a law of perception or a guide for isolated judgements?

The literature agrees that encodings differ in how accurately they carry values and disagrees about how far that difference governs design. One position treats the ordering as a property of the visual system that can be built into a rule. Mackinlay's (1986) automatic designer made effectiveness an ordering of visual variables and chose an encoding from it, following the guideline Cleveland and McGill (1984) drew.

The opposing position treats the ordering as holding only for isolated abstract judgements. Simkin and Hastie (1987) found the display interacting with the task. Hollands and Spence (1992) found change and proportion judgements drawing on different strategies. Shah and Freedman (2011) found prior beliefs entering comprehension.

What would settle it is two findings:

1. A study that assigns channels within a real analytical task and measures the quality of the decision reached, rather than the accuracy of reading an isolated value.
2. A study that reports how channels combine, rather than holding them fixed one at a time, so that the interaction Heer and Bostock (2010) warn about is measured instead of assumed away.

Neither is routine, which is why the dispute stays open. Franconeri and colleagues (2021) set out the distance that remains, finding the evidence strong on encoding and annotation but a gap between laboratory judgement and the real decisions displays are meant to serve.

The two positions rest on different evidence. The law position rests on controlled judgement experiments with abstract stimuli, and the guide position on task experiments, fewer in number. While the dispute is open, use the ranking to shortlist encodings for a named reading, then test the finished display against that reading, and treat a channel's rank as a tendency rather than a guarantee.

The strongest counter-argument

The strongest objection is that the task findings do not weaken the ranking, because a competent designer never used it as a whole-display instruction. The ranking is comparative: use the highest task that will carry the reading you need, putting the value comparison on position and reserving colour for grouping.

The reply is that this concedes the claim rather than answering it. The counter-argument describes what a designer does when a ranking is insufficient: that designer decides which channel serves which judgement, and that decision is where the task and the reader's prior beliefs enter. The claim is not that the ranking is false but that it is incomplete.

Where the area is heading

The direction is from ranking channels in isolation to testing displays in the decisions they serve. Franconeri and colleagues (2021) fold uncertainty and annotation into the same account and name the laboratory-to-decision gap as the open problem. Ware (2020) supplies the perceptual rationale any such test has to respect, and Healey, Booth and Enns (1996) measured the preattentive estimation that rationale rests on. When a display is built to inform a decision rather than to be read, what to measure stays open.

The sources this unit rests on

Each note states what the cited document did and found rather than quoting it, from the resolved metadata and abstracts of the records in the module plan.

• Bertin, J. (2011). Semiology of Graphics (W. J. Berg, Trans.). https://www.esri.com/en-us/esri-press/browse/semiology-of-graphics-diagrams-networks-maps Proposed a theory of graphic signs in which a variable may encode data only if it has the corresponding perceptual property, so ordered variables carry ordered data and selective variables carry categories, the constraint the ranking later measured.
• Borkin, M. A., Vo, A. A., Bylinskii, Z., et al. (2013). What Makes a Visualization Memorable? doi:10.1109/tvcg.2013.234 Found memorability a stable property of the image, only weakly related to chart type or aesthetic rating, with colour, density and recognisable elements predicting recall, so a display has properties beyond accuracy.
• Cleveland, W. S. and McGill, R. (1984). Graphical Perception: Theory, Experimentation, and Application to the Development of Graphical Methods. doi:10.1080/01621459.1984.10478080 Had readers judge the relative size of two encoded values, and the pattern of errors ordered elementary perceptual tasks, with position on a common scale and length most accurate and area, angle and colour saturation least. It is the measurement the unit tests.
• Cleveland, W. S. and McGill, R. (1986). An experiment in graphical perception. doi:10.1016/s0020-7373(86)80019-0 Ran a further experiment on six basic judgements and recovered the ordering, with the two position judgements most accurate, length second, angle and slope third and area last. Distance between the judged objects affected accuracy and technical training did not.
• Franconeri, S. L., Padilla, L. M., Shah, P., et al. (2021). The Science of Visual Data Communication: What Works. doi:10.1177/15291006211051956 Synthesised perceptual and cognitive research on reading data displays, covering encoding, annotation and uncertainty, and concluded that several widely repeated design intuitions are unsupported. It states where the perceptual consensus stands and where it stops short of design advice.
• Heer, J. and Bostock, M. (2010). Crowdsourcing graphical perception: using mechanical turk to assess visualization design. doi:10.1145/1753326.1753357 Replicated the classic judgement experiments with crowdworkers using the same error measure and found the shape and order preserved; a new experiment found squares and one-to-one aspect ratios gave the worst area judgements, explained by a one-dimensional length proxy, so it carries the ranking's limits.
• Hollands, J. G. and Spence, I. (1992). Judgments of Change and Proportion in Graphical Perception. doi:10.1177/001872089203400306 Across two experiments compared how accurately readers judged change and proportion from line, bar and pie forms and found the two drew on different perceptual strategies, so a form good for one was not good for the other.
• Lewandowsky, S. and Spence, I. (1989). The perception of statistical graphs. doi:10.1177/0049124189018002002 Reviewed the psychological evidence on graph reading and reported reliable differences among encodings for particular judgements while finding the evidence base thinner than the field assumed. It dates the unit's disagreement to before the ranking became canonical.
• Mackinlay, J. (1986). Automating the design of graphical presentations of relational information. doi:10.1145/22949.22950 Defined a language for graphical presentations with expressiveness and effectiveness criteria and built a system that selected an encoding from a ranked list of perceptual codes, treating effectiveness as an ordering of visual variables. It is where the ranking became a rule inside a program, inheriting whatever it had not measured.
• Quadri, G. J., Wang, A. Z., Wang, Z., Adorno, J., Rosen, P. and Szafir, D. A. (2024). Do You See What I See? A Qualitative Study Eliciting High-Level Visualization Comprehension. doi:10.1145/3613904.3642813 Had participants describe line graphs, bar graphs and scatterplots in think-aloud protocols and found that the statistics estimated in classical perception studies differ from the high-level patterns readers comprehend, and that chart type alone does not predict what they extract.
• Shah, P. and Freedman, E. G. (2011). Bar and Line Graph Comprehension: An Interaction of Top-Down and Bottom-Up Processes. doi:10.1111/j.1756-8765.2009.01066.x Modelled comprehension as an interaction between bottom-up perceptual encoding and top-down knowledge and expectation and found prior beliefs and graph form jointly determined what readers took from a display. It shows the ranking describes only the bottom-up half of reading a chart.
• Simkin, D. and Hastie, R. (1987). An Information-Processing Analysis of Graph Perception. doi:10.1080/01621459.1987.10478448 Analysed graph reading as a sequence of component operations and found display type interacted with task, so no single form was best, and argued that design should follow the component processes a reader must execute. It is the early task-dependence evidence set against a universal ranking.
• Spence, I. and Lewandowsky, S. (1991). Displaying proportions and percentages. doi:10.1002/acp.2350050106 Compared pie charts, divided bars and other forms for reading proportions and found the form made little difference for some judgements while errors were large for others, concluding that the display should be matched to the particular judgement required. It grounds the unit's treatment of pie charts.
• Ware, C. (2020). Information Visualization: Perception for Design (4th ed.). https://shop.elsevier.com/books/information-visualization/ware/978-0-12-812875-6 Set out the perceptual and cognitive science relevant to information display, covering colour, attention and pattern perception, and argued that display design should follow the properties of the visual system rather than convention.
• Healey, C. G., Booth, K. S. and Enns, J. T. (1996). High-speed visual estimation using preattentive processing. doi:10.1145/230562.230563 Found that rapid and accurate estimation of a proportion was possible when the target was defined by hue or orientation, with boundary conditions set by display duration and feature difference.
• Zacks, J. and Tversky, B. (1999). Bars and lines: A study of graphic communication. doi:10.3758/bf03201236 Asked participants to describe and draw bar and line graphs and found readers took lines as trends over time and bars as discrete comparisons, so the form shaped the conceptual interpretation of the data. It shows encoding changes what a reader believes the data are about.