Screen readers, semantics and assistive-technology interoperability

A page is two objects. One is the picture its author sees: boxes, colours, a heading that looks like a heading. The other is the accessibility tree, the structure of roles, names, values and states that a browser exposes and a screen reader reads. Whether a page works is settled in the second object and in the interaction model built on it.

This seventh of ten units follows the software the sixth described, and argues that a failure of the tree is invisible to the people who build the page.

A screen reader must model the page before it speaks it

Asakawa and Itoh (1998) described the interface of a Home Page Reader, a system that turned a visual page into a spoken one. It had to decide what a page was made of before deciding what to say. That decision is the accessibility tree: each element becomes a node with a role, a name and a state.

The interaction model on the tree is not the visual one. Borodin, Bigham, Dausch and Ramakrishnan (2010) surveyed the browsing strategies screen reader users employ, from skimming and jumping between landmarks to coping with content that refreshes without warning. They found a repertoire that differs sharply from sighted browsing. Kane, Bigham and Wobbrock (2008) turned that difference into an interface. Their Slide Rule placed a touch screen's controls on multi-touch gestures with audio feedback. In a study with ten blind participants it was significantly faster than a button-based screen reader, and seven of the ten preferred it. Participants made more errors than with the familiar button system, the honest cost of speed on an unfamiliar interaction model.

The practice is diverse, which any single test hides. WebAIM (2024) received 1,539 valid responses, 20.0 per cent of them from people aged 60 or over. JAWS was the primary desktop reader for 40.5 per cent and NVDA for 37.7 per cent. In total, 71.6 per cent used more than one desktop reader.

Navigation is structure, not decoration

The tree is where a user finds things, so its shape sets the cost of every task. 71.6 per cent of respondents navigate through the headings first on a lengthy page, and 88.8 per cent found heading levels very or somewhat useful (WebAIM, 2024). Only 3.7 per cent used landmarks first. Headings styled to look like body text remove the route most users take, and the one a reader in a second language needs, without changing a pixel.

Structure can be supplied after the fact, and the gain is large. Takagi, Asakawa, Fukuda and Maeda (2002) showed that site-wide annotations improved navigability for blind users without editing every page by hand. Srinivasan, Harshbarger, Hilliker and Mankoff (2023) built Azimuth through co-design with two screen reader users and evaluated it with five. Presenting data as navigable structure rather than a text stream supported both targeted and open-ended analysis.

Content that never reaches the tree

Correct semantics cannot help content that was never described. Bigham, Kaminsky, Ladner, Danielsson and Hempton (2006) found a large fraction of significant images carried no alternative text. They built WebInSight to generate labels, routing images for human description when generation failed.

Two decades on, the gap has not closed. Oh, Joh and Lee (2021) reviewed 33 papers on touchscreen image accessibility. They found most approaches generated image information by hand or assumed it, automation little studied, and users mostly involved only in evaluation.

Why the people who build a page cannot see the failure

Those who can check a page by eye are the least equipped to detect a failure that leaves no visual trace. Leporini and Paterno (2008) identified fifteen design criteria that integrated usability with accessibility for vision-impaired users. The criteria reduced navigation time and made the sites easier to navigate, so accessibility guidelines on their own had not guaranteed a usable page.

Semantics carries the trap in a sharper form: the mistake is often an addition rather than an omission. Faulkner and MacDonald (2018) set out five rules of ARIA use. The first is to use a native element that already carries the required semantics and behaviour before adding a role or state:

If you can use a native HTML element ... with the semantics ... already built in, instead of re-purposing an element and adding an ARIA role, state or property to make it accessible, then do so.

A role added to an element that did not need one does not fix the tree. It makes the tree report something false, invisibly.

Access decays, and the user pays for it in repair time

Morris and colleagues (2016) studied blind users of a social platform as it moved towards images and dynamic content. They documented previously usable access being eroded by that shift.

Shinohara and Tenenberg (2009) gave a first-hand account of one blind person's technology, the substitutes and repairs it required, and where mainstream tools failed.

Lazar, Allen, Kleinman and Malarkey (2007) collected the frustrations of 100 blind screen reader users. The leading complaint was page layout that produced confusing screen reader feedback, ahead of unlabelled forms and missing image text. Respondents reported that they lost on average 30.4 per cent of their time to such frustrations. The failures that cost most time are rarely the ones a visual check catches.

Contested question: is the fault in the tree, or in what the field studies?

The literature agrees that the tree is where a page works or fails. It disagrees about why the failures persist. One position holds that the fault is a technical defect better tools will close, because the tree lacks labels, roles and structure that could be supplied. Bigham and colleagues (2006) automated alternative text, Oh, Joh and Lee (2021) set out a research road map, and Faulkner and MacDonald (2018) treat correct semantics as an authoring discipline.

The opposing position holds that the defect is in the field's choices rather than its tooling. Mack and colleagues (2021) coded accessibility papers from CHI and ASSETS. Over 43 per cent of the recent ones concerned blind and low-vision people, and disabled people were often evaluated rather than asked the research question. Oh, Joh and Lee (2021) record the same pattern inside image accessibility. A third position holds that both accounts understate the maintenance the user performs, because Shinohara and Tenenberg (2009) and Morris and colleagues (2016) show access sustained by constant repair against systems that keep changing.

The two sides rest on evidence of different kinds. The technical-defect case rests on a system built to close one defect class, and on a prototype co-designed with two screen reader users and evaluated with five. Together they show that the defect can be addressed, rather than that addressing it helps at scale. The field-choices case rests on a literature survey that coded 506 papers, which measures what the field studies rather than what the tree lacks. The maintenance case rests on two interview studies.

The strongest objection is that the tree is not where the user works. The user runs a reader, a browser and a set of habits, and the unit's own survey shows that 71.6 per cent of respondents use more than one desktop reader. A page that fails in the tree can still be completed through a workaround, and a page that passes can still fail in use. A cleaner tree is therefore a necessary condition whose sufficiency the unit assumes.

What would settle it is a comparison the literature lacks. It would take one named defect class, close it with the best available automation, and measure screen reader users' task performance and time on the fixed pages against matched pages that were never fixed. Lazar and colleagues (2007) supply the time and frustration measures and Srinivasan and colleagues (2023) the comprehension measures.

What to do while the dispute is open

While the dispute is open, test with the reader and the workflow the user actually runs. Record every failure against the node that caused it, and treat the name, the role and the state as the artefact under test.

Where the argument is heading

The named work points the same way: towards treating the tree as a designed artefact and its users as designers. Srinivasan and colleagues (2023) built navigable structure through co-design. Oh, Joh and Lee (2021) make automation and early user involvement the two items of their road map. Mack and colleagues (2021) ask the field to change which questions it counts as accessibility research. The next unit moves below the interface to cognition, neurodiversity and ageing, where the same tree meets needs that resist a testable criterion.

The sources this unit rests on

• Asakawa, C. and Itoh, T. (1998). User interface of a Home Page Reader. Proceedings of the Third International ACM Conference on Assistive Technologies. doi:10.1145/274497.274526 Described the interface and interaction model of a Home Page Reader that turned a visual page into a spoken one. It is the unit's starting point: a screen reader must model a page before it can speak it.
• Takagi, H., Asakawa, C., Fukuda, K. and Maeda, J. (2002). Site-wide annotation: reconstructing existing pages to be accessible. Proceedings of the Fifth International ACM Conference on Assistive Technologies (ASSETS). doi:10.1145/638249.638265 Reconstructed existing pages with site-wide annotations and improved their navigability for blind users without hand-editing every page. Structure, not content alone, carries much of the barrier and can be supplied.
• Bigham, J. P., Kaminsky, R. S., Ladner, R. E., Danielsson, O. M. and Hempton, G. L. (2006). WebInSight: making web images accessible. Proceedings of the 8th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS). doi:10.1145/1168987.1169018 Found that a large fraction of significant images carried no alternative text and built a system to generate labels, with human labelling as fallback. The fallback shows how common the omission was.
• Lazar, J., Allen, A., Kleinman, J. and Malarkey, C. (2007). What Frustrates Screen Reader Users on the Web: A Study of 100 Blind Users. International Journal of Human-Computer Interaction. doi:10.1080/10447310709336964 Recorded the frustrations of 100 blind screen reader users; the leading complaint was page layout that produced confusing screen reader feedback, with unlabelled forms and missing image text also among the top causes and respondents reporting an average loss of 30.4 per cent of their time. These are the failures that cost users time.
• Leporini, B. and Paterno, F. (2008). Applying Web Usability Criteria for Vision-Impaired Users: Does It Really Improve Task Performance? International Journal of Human-Computer Interaction. doi:10.1080/10447310701771472 Identified fifteen design criteria integrating accessibility and usability for vision-impaired users and tested them in two user studies; the criteria reduced navigation time and made the sites easier to navigate. Accessibility guidelines alone had not guaranteed usability.
• Shinohara, K. and Tenenberg, J. (2009). A blind person's interactions with technology. Communications of the ACM. doi:10.1145/1536616.1536636 Gave a first-hand account of one blind person's technology use, the workarounds required and where mainstream tools fail. It documents the repair work that keeps access running.
• Borodin, Y., Bigham, J. P., Dausch, G. and Ramakrishnan, I. V. (2010). More than meets the eye: a survey of screen-reader browsing strategies. Proceedings of the 2010 International Cross-Disciplinary Conference on Web Accessibility (W4A). doi:10.1145/1805986.1806005 Surveyed the browsing strategies screen reader users employ on difficult pages and showed they differ sharply from sighted browsing. It is the unit's account of the distinct interaction model the tree supports.
• Kane, S. K., Bigham, J. P. and Wobbrock, J. O. (2008). Slide rule: making mobile touch screens accessible to blind people using multi-touch interaction techniques. Proceedings of the 10th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS). doi:10.1145/1414471.1414487 Introduced audio-based multi-touch techniques that let blind users operate a touch screen without locating objects visually. A study with ten blind participants found Slide Rule significantly faster than a button-based screen reader and preferred by seven of ten, but with more errors.
• Morris, M. R., Zolyomi, A., Yao, C., et al. (2016). With most of it being pictures now, I rarely use it: understanding Twitter's evolving accessibility to blind users. Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. doi:10.1145/2858036.2858116 Studied blind users of a platform as it shifted towards images and dynamic content and found previously usable access eroded. Access decays as a system evolves, without anyone removing it.
• Faulkner, S. and MacDonald, D. (2018). Using ARIA. W3C Working Draft, 27 September 2018, World Wide Web Consortium. https://www.w3.org/TR/using-aria/ Set out five rules of ARIA use, the first being to use a native element that already carries the required semantics and behaviour rather than re-purposing an element and adding a role. W3C retired the document on 24 February 2026, keeping four rules for historical reference.
• Mack, K., McDonnell, E., Jain, D., et al. (2021). What do we mean by 'accessibility research'? A literature survey of accessibility papers in CHI and ASSETS from 1994 to 2019. Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. doi:10.1145/3411764.3445412 Coded 506 papers from 2010 to 2019 and counted 836 across 26 years; over 43 per cent of the recent papers concerned blind and low-vision people, and median participant groups with disabilities or older adults numbered 13. The field's agenda is narrower than its object.
• Oh, U., Joh, H. and Lee, Y. (2021). Image accessibility for screen reader users: a systematic review and a road map. Electronics. doi:10.3390/electronics10080953 Reviewed 33 papers on touchscreen image accessibility and found most approaches generated image information manually or assumed it, automation little studied, and users mostly involved only in evaluation. The oldest content barrier is unresolved.
• Srinivasan, A., Harshbarger, T., Hilliker, D. and Mankoff, J. (2023). Azimuth: designing accessible dashboards for screen reader users. Proceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS). doi:10.1145/3597638.3608405 Co-designed with two screen reader users and evaluated with five, generating dashboards whose data is navigable structure rather than a text stream, which supported targeted and open-ended analysis. The tree can be designed, not merely repaired.
• WebAIM (2024). Screen Reader User Survey #10 Results. WebAIM, Utah State University. https://webaim.org/projects/screenreadersurvey10/ Surveyed screen reader users in December 2023 and January 2024 with 1,539 valid responses; JAWS was the primary desktop reader for 40.5 per cent and NVDA for 37.7 per cent, against 53.7 and 30.7 per cent in 2021, and 71.6 per cent used more than one desktop reader. It replaces Survey #9 as the current account of which readers users run.