Ability-based design and the adaptive interface

A standard keyboard and mouse impose a measurable cost on some users, and the measurement predates the term ability-based design. Trewin and Pain (1999) observed 20 computer users with motor disabilities and six without. They identified six classes of keyboard difficulty that cost significant correction time, and recorded failures across all aspects of mouse use. Their closing observation is the hinge: many of those difficulties would be amenable to automatic detection. Nearly twenty years later, Mott and colleagues (2018) surveyed 46 people with motor impairments and interviewed 12 about smartphone photography. They took fewer photographs than they wanted, improvised around interactions that assume steady control, and judged their own images poor.

If the machine can sense the error, it can respond, and the person can be left as they are. Ability-based design is the field's answer. This third of ten units argues that it is the strongest technical idea in accessible computing, and that its evidence base is narrower than its principles. The benefit tracks the stability of what the system senses: largest for a measured motor channel such as pointing throughput, smallest where the system must predict a preference. The approach is oversold less by its careful authors than by the distance between its seven principles and the studies beneath them.

The failures were repeatable, so adaptation became an optimisation

The failures Trewin and Pain recorded are behavioural and repeatable, so they can be modelled. Gajos and Weld (2004) proposed SUPPLE, which treats adaptation as an optimisation problem. It searches for the rendition that meets a device's constraints and minimises the estimated effort of the user's expected actions.

Wobbrock, Myers and Hsu (2007) widened the input. SUPPLE++ infers motor capabilities from a one-time performance test and adjusts for low vision as well as motor impairment. One participant completed tasks she could not perform with a standard interface. The rest saved an average of 20 per cent in time, from a 3 per cent slowdown to a 43 per cent speedup.

The engine scales to run time, and its authors state the boundary

Gajos, Weld and Wobbrock (2010) made the approach an engineering result. They showed that generation over a solution space of up to ten to the power of seventeen interfaces is feasible at run time. The paper also states the boundary plainly: Supple is not intended to replace human designers, only to offer alternatives for the people the hand-made designs do not serve. Sarcar and colleagues (2017) carried the same optimisation to a second ability group. They tuned a smartphone text-entry layout to the finger tremor that ageing brings, taking the parameters that describe pointing as the design variables. They reported that the tuned interface improved touch-typing accuracy over the standard Qwerty keyboard for users with tremor. It is an initial study, so it shows the method travels rather than that it holds at scale or runs on the second-hand device a user already owns.

The strongest number says the adaptation works

In Gajos, Wobbrock and Weld (2008), participants with motor impairments using SUPPLE++ interfaces were 26.4 per cent faster and made 73 per cent fewer errors. They strongly preferred them to manufacturers' defaults, and they rated them easier and much less tiring.

The angle mouse (Wobbrock et al., 2009) shows the adaptation need not touch the layout. It adjusts mouse gain from the angular deviation of the user's own movement, so it is target-agnostic and never adapts around the wrong thing. In a study of 16 people, eight with motor impairments, motor-impaired pointing throughput improved by 10.3 per cent over the Windows default and 11.0 per cent over sticky icons. Able-bodied throughput stayed within 1.2 per cent of the default.

The principles ask for less than the name implies

Writers cite ability-based design through its seven principles, and most often drop their structure. Wobbrock and colleagues (2011) group them into stance (Ability and Accountability), interface (Adaptation and Transparency) and system (Performance and Context), with Commodity an encouraged seventh. Accountability asks designers to respond to poor performance by changing the system rather than the person, which is this unit's thesis written as an instruction. The field's own definition therefore does not require a system to adapt: a design that removes the able-bodied assumption can comply.

A self-changing interface can be slower than a fixed one

Findlater and McGrenere (2004) compared three split menus with 27 participants: a static menu, an adaptable menu the participant arranged, and an adaptive menu the system rearranged. The static menu was significantly faster than the adaptive one, and the adaptable menu was significantly faster under certain conditions. The automatic condition predicts preferences rather than measuring ability.

Lavie and Meyer (2010) give the caution a scope. The benefits depended on task frequency, user age, task difficulty and user involvement. A fully adaptive system helped in familiar, routine situations, particularly for older participants, while in unfamiliar situations cognitive workload rose substantially and performance fell. Hurst and Tobias (2011) add that purchased assistive technology is abandoned at rates of 35 per cent or more. They found that control over design elements motivated people to build their own devices.

Contested question: does the system or the user hold the authority to adapt?

The literature disagrees about who should decide, and both positions have evidence. The system-authority position holds that the system should adapt. It can measure what the user cannot easily report, and the gains are large. Wobbrock and colleagues (2011) offer the principle of changing the system rather than the user.

The user-authority position holds that the person should direct the adaptation, because the system's model of the person is exactly what is uncertain. Findlater and McGrenere (2004) found the user-controlled menu faster and better liked. Lavie and Meyer (2010) found automatic adaptation helped only in routine situations. Hurst and Tobias (2011) found control over design decisions was itself a reason people adopted a technology.

The two sides rest on evidence of different kinds, and the kinds are not measuring the same adaptation. The system case rests on a controlled study in which participants with motor impairments were 26.4 per cent faster and made 73 per cent fewer errors, and on a second study of target-agnostic gain adjustment. Both tune a stable motor channel. The user case rests on a controlled menu study with 27 participants and on one study of four adaptivity levels in a vehicle, and both turn on a system predicting a preference. The first measures ability; the second measures prediction.

The strongest objection is that the measured gains may not survive the field. The demonstrations used short tasks with participants recruited for a study. The one field study in the unit reports how badly a standard interface fits, rather than how well an adapted one holds up over weeks. The strongest technical idea in the unit has no field trial behind it.

What would settle it is a comparison the literature has not run. It would put the same users, tasks and interface under automatic adaptation, then under user-directed adaptation with full transparency and reversal, over weeks of ordinary use on a slow connection, and report throughput, errors, abandonment and model accuracy. Mott and colleagues (2018) show the field-study half of that design.

What to do while the dispute is open

While the dispute is open, make the system's model of the user inspectable and let the user override it. Adapt only on a quantity the system measures, say which quantity that is, and keep a fixed setting one action away.

Where the argument is heading

The named work points at measurement rather than more prototypes. Mitchell and Wobbrock (2024) review how motor ability has been quantified. They conclude that the construct is surprisingly elusive and absent from existing accounts of ability-based design. Authors borrow disparate definitions, with no agreement on what is measured. They propose a three-dimensional framework of user, context and task. If the benefit of adaptation depends on how well the system has modelled the person, the model is what needs validating. Wobbrock and colleagues (2018) generalised the approach beyond accessibility. The coming work is to make ability measurable and to decide, on evidence, how much of the adaptation the user should direct.

The sources this unit rests on

• Trewin, S. and Pain, H. (1999). Keyboard and mouse errors due to motor disabilities. International Journal of Human-Computer Studies. doi:10.1006/ijhc.1998.0238 Studied 20 computer users with motor disabilities and six without, identifying six classes of keyboard difficulty and difficulties across all aspects of mouse use, particularly pointing and dragging. It notes that many of these would be amenable to automatic detection.
• Gajos, K. and Weld, D. S. (2004). SUPPLE: automatically generating user interfaces. Proceedings of the 9th International Conference on Intelligent User Interfaces (IUI). doi:10.1145/964442.964461 Treats interface adaptation as an optimisation problem: the system searches for the rendition meeting a device's constraints that minimises the estimated effort of the user's expected actions. It is the technical foundation of generated, ability-matched layouts.
• Findlater, L. and McGrenere, J. (2004). A comparison of static, adaptive, and adaptable menus. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. doi:10.1145/985692.985704 A controlled study with 27 participants found the static menu significantly faster than the adaptive one, and the adaptable menu significantly faster under certain conditions, with a majority preferring the adaptable menu. It shows that a self-changing interface can be slower than a fixed one.
• Wobbrock, J. O., Myers, B. A. and Hsu, H. H. S. (2007). Automatically generating user interfaces adapted to users' motor and vision capabilities. Proceedings of the 20th annual ACM symposium on User interface software and technology. doi:10.1145/1294211.1294253 SUPPLE++ models motor capabilities from a one-time performance test and generates a personalised interface that adjusts for low vision as well as motor impairment. One user completed tasks she could not perform with a standard interface, while the rest averaged a 20 per cent time saving, from a 3 per cent slowdown to a 43 per cent speedup.
• Gajos, K. Z., Wobbrock, J. O. and Weld, D. S. (2008). Improving the performance of motor-impaired users with automatically-generated, ability-based interfaces. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. doi:10.1145/1357054.1357250 Participants with motor impairments were 26.4 per cent faster and made 73 per cent fewer errors with generated ability-based interfaces, and preferred them to manufacturers' defaults. It is the clearest controlled evidence that the system can absorb the user's cost.
• Wobbrock, J. O., Fogarty, J., Liu, S.-Y. (S.), Kimuro, S. and Harada, S. (2009). The angle mouse: target-agnostic dynamic gain adjustment based on angular deviation. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI). doi:10.1145/1518701.1518912 The angle mouse adjusts mouse gain from the angular deviation sampled during movement, so it needs no knowledge of target locations. In a study of 16 people, eight with motor impairments, it improved motor-impaired pointing throughput by 10.3 per cent over the Windows default and 11.0 per cent over sticky icons, with no significant cost to able-bodied users.
• Gajos, K. Z., Weld, D. S. and Wobbrock, J. O. (2010). Automatically generating personalized user interfaces with Supple. Artificial Intelligence. doi:10.1016/j.artint.2010.05.005 Formalises interface generation as an optimisation problem over a solution space of up to ten to the power of seventeen interfaces, showing that exact solutions arrive in under a second in most cases. It reports improved speed, accuracy and satisfaction for people with motor impairments, and states that Supple is not intended to replace human designers.
• Sarcar, S., Jokinen, J. P. P., Oulasvirta, A., Wang, Z., Silpasuwanchai, C. and Ren, X. (2017). Ability-based optimization: designing smartphone text entry interface for older adults. Human-Computer Interaction - INTERACT 2017. doi:10.1007/978-3-319-68059-0_22 Extends model-based interface optimisation to ability-based design for older adults, tuning a text-entry layout to finger tremor and reporting improved touch-typing accuracy over Qwerty in an initial study. It tests whether the method travels to a second ability.
• Lavie, T. and Meyer, J. (2010). Benefits and costs of adaptive user interfaces. International Journal of Human-Computer Studies. doi:10.1016/j.ijhcs.2010.01.004 Studied an in-vehicle telematic system across four levels of adaptivity and found that benefits depended on task frequency, user age, task difficulty and user involvement. A fully adaptive system helped in familiar, routine situations, while cognitive workload rose substantially in unfamiliar ones.
• Hurst, A. and Tobias, J. (2011). Empowering individuals with do-it-yourself assistive technology. The proceedings of the 13th international ACM SIGACCESS conference on Computers and accessibility. doi:10.1145/2049536.2049541 Reports that 35 per cent or more of purchased assistive technology devices end up unused or abandoned. It finds that control over design elements, along with passion and cost, motivated people to build their own technology rather than buy it.
• Wobbrock, J. O., Kane, S. K., Gajos, K. Z., Harada, S. and Froehlich, J. (2011). Ability-based design: concept, principles and examples. ACM Transactions on Accessible Computing. doi:10.1145/1952383.1952384 Proposes ability-based design, which makes ability rather than disability the focus of the design process, and sets out seven principles in three groups: stance, interface and system. Only the first two, ability and accountability, are required for a design to count as ability-based.
• Wobbrock, J. O., Gajos, K. Z., Kane, S. K. and Vanderheiden, G. C. (2018). Ability-based design. Communications of the ACM. doi:10.1145/3148051 Restates ability-based design for a general computing readership as a focus on what people can do rather than what they cannot, and on systems adapting to users rather than the other way around. It generalises the approach beyond accessibility, arguing that designing for ability produces better systems for everyone.
• Mott, M. E., Bennett, C. L., Cutrell, E. and Morris, M. R. (2018). Understanding the accessibility of smartphone photography for people with motor impairments. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. doi:10.1145/3173574.3174094 Surveyed 46 people with motor impairments and interviewed 12, finding that they took fewer photographs than they wanted and improvised around interactions that assume steady control. Two alternative photography interfaces were tested as design probes.
• Mitchell, C. L. and Wobbrock, J. O. (2024). Characterizing 'motor ability' for ability-based design. The 26th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS). doi:10.1145/3663548.3675646 Reviews the literature that quantifies motor abilities and argues that the construct of ability is surprisingly elusive and absent from existing accounts of ability-based design, with disparate definitions borrowed without agreement on what is measured. It proposes a three-dimensional framework of user, context and task.