Mobile interaction is shaped by one hand, a moving body and a millimetre-scale target

Parhi, Karlson and Bederson (2006) measured one-handed thumb use on small touchscreens. They recommended targets of 9.2 mm for discrete tasks and 9.6 mm for serial ones. The constraints of mobile use are the design conditions rather than edge cases to accommodate afterwards, and this fifth of ten units returns from the model to the body holding the device. They are one hand, a moving body, situational impairment and a target a few millimetres across. If speed and accuracy change with grip, posture and movement, one interaction design cannot serve all the ways a phone is held.

The hand sets the target size before the designer does

Parhi and colleagues (2006) ran the measurement in two phases, discrete pointing at a single target and serial entry across a sequence. In the discrete phase, error rates stopped differing significantly once targets reached about 9.6 mm. In the serial phase the threshold was about 7.7 mm, while speed improved as targets grew. Target position also changed performance, because a device held in one hand constrains where the thumb can travel.

The constraint also has a built design answer. Karlson and Bederson (2007) built ThumbSpace. It puts a scaled copy of the whole screen under the thumb, so a far target becomes a short reach. The comparison with direct touch shows that reach is bought with time. Direct touch was much faster, 811 milliseconds against 2068. ThumbSpace was more accurate outside thumb reach, at 94 per cent against 92, and it made small targets as easy to hit as large ones. Most participants still preferred direct touch, though 69 per cent expected to prefer ThumbSpace with practice.

The error is systematic, and a correction recovers part of it

Henze, Rukzio and Boll (2011) measured the same constraint without a laboratory. They published a game in the Android Market that collected 120,626,225 touch events from 91,731 installations. Target size and screen location systematically skewed the touch positions. A compensation function derived from the data cut the error rate by 7.79 per cent. It did so in an independent test of 15,326,444 further touches. Bergstrom-Lehtovirta and Oulasvirta (2014) supplied the geometry behind both results. They derived the thumb's functional area from the kinematics of the gripping hand, and fitted it to trajectories from 20 participants. The model predicts the reachable area from surface size, hand size and index finger position.

Grip and posture move the number

Azenkot and Zhai (2012) had 32 participants enter text in three postures on soft QWERTY keyboards: two thumbs, one thumb and one index finger. Accuracy and precision differed systematically between the postures, with consistent touch offsets that an algorithm could correct. Keyboards should therefore adapt to the posture in use.

Wobbrock, Myers and Aung (2008) put the posture effect into Fitts' law terms. Two-handed postures were faster than one-handed ones, at 1360 milliseconds against 1575, and index fingers were faster than thumbs, at 1408 against 1527. The index finger on the back of the device outperformed the thumb on the front in horizontal movements. It was worse at complex gestures than a front index finger.

The interaction is shorter than the task

Bohmer and colleagues (2011) logged detailed application usage from over 4,100 Android users. The average session with an application lasted less than a minute even though users spent almost an hour a day on the phone. The first application opened when the device woke was almost always a communication application. Oulasvirta and colleagues (2011) describe the mechanism, a checking habit of brief, repetitive inspection of quickly accessible dynamic content. Their field experiment showed checking behaviour reinforced by informational rewards that arrive immediately. Their qualitative data show habitual use experienced as an annoyance rather than an addiction.

Text entry makes the constraint set measurable, at rest and on the move

Typing puts every constraint on one surface. Palin and colleagues (2019) collected data from 37,370 volunteers in a web transcription task. They reported a mean speed of 36.2 words per minute with 2.3 per cent uncorrected errors. A person typing in a second language on a predictive keyboard is part of the distribution, not an exception. Performance varied with age and finger usage, and auto-correct use correlated positively with entry rate while word prediction correlated negatively. Dhakal and colleagues (2018) analysed 136 million keystrokes from 168,000 volunteers. They found rollover typing, pressing the next key before releasing the previous one, surprisingly prevalent. Their clustering on inter-key intervals separates most users into eight groups differing in speed, accuracy, hand and finger usage and rollover.

Goel, Findlater and Wobbrock (2012) built WalkType, an adaptive text entry system. It reads the phone's accelerometer and corrects keystroke interpretation while the user walks, using displacement, acceleration and inferred footsteps from 16 participants. Walking participants showed 45.2 per cent fewer uncorrected errors and typed 12.9 per cent faster than in the control condition. Movement degrades a screen task measurably, and a sensor inside the device can recover part of the loss.

One design cannot serve every hand or every user

Kane, Wobbrock and Ladner (2011) asked ten blind and ten sighted people to invent gestures for common computing tasks and then perform a reference set. Blind participants preferred edge-based gestures and gestures that tap virtual keys. Their performance differed significantly in speed, size and shape, so a gesture set tuned to sighted users underperforms for the people it leaves out. Hassan and colleagues (2023) widen the channel itself: 12 Deaf participants entered text by fingerspelling on a smartphone. With practice, they reached 42.5 words per minute against 31.9 for a virtual keyboard. Their corrected error rates were 4.02 per cent against 6.46 per cent. A second study preferred signing and the keyboard over fingerspelling for walk-up use.

The contested question: compensate, constrain, or configure?

Three positions are held in this literature, and they disagree about where the problem should be fixed. The first holds that the interface should compensate in software. A correction function cut touch error by 7.79 per cent (Henze et al., 2011). Accelerometer data cut walking error by 45.2 per cent (Goel et al., 2012). Keyboards adapt to the posture in use (Azenkot and Zhai, 2012). The remedy is a sensing interface, and success is a lower error rate on the same targets.

The second holds that the interface should stay inside the measured envelope. Parhi and colleagues (2006) supply the target floor. Bergstrom-Lehtovirta and Oulasvirta (2014) supply the reachable area for a given hand, and Wobbrock and colleagues (2008) the posture that performs best. The remedy is fewer, larger, reachable controls and an explicit posture assumption, and success is fewer errors without sensing anything.

The third holds that there is no single design target at all. Kane and colleagues (2011) show a sighted gesture set failing blind users. Hassan and colleagues (2023) show a mainstream entry method outperformed by another channel for Deaf users. Dhakal and colleagues (2018) show the population splitting into eight groups of typists. The remedy is per-user configuration, and success is parity across groups rather than a better average.

What would settle the choice is a comparison the literature does not contain. One study would run all three strategies on the same tasks. It would log grip and movement on the same touch events, and report results per posture group and per user group. The obstacle is that logging the grip both enables adaptation and turns the trace into profiling data (Lemarquis et al., 2023).

Rated by kind, the compensation side rests on large-scale touch logs and controlled text-entry studies (Henze et al., 2011; Goel et al., 2012; Azenkot and Zhai, 2012), so it carries strong evidence. The constraint side rests on measurement studies of target size and reach (Parhi et al., 2006; Bergstrom-Lehtovirta and Oulasvirta, 2014; Wobbrock et al., 2008). The configure side rests on comparisons between user groups (Kane et al., 2011; Hassan et al., 2023) and on one cluster analysis (Dhakal et al., 2018). While the dispute is open, a designer picks the largest reachable targets the layout holds and adds compensation only where the sensing is acceptable. The strongest objection is that the sensing adaptation needs may identify the user.

Where this is heading

Hassan and colleagues (2023) show the entry channel widening beyond tapping and swiping. Lemarquis, Yilma and Leiva (2023) show the price of the sensing that adaptation needs. Recurrent classifiers reading swipe trajectories predicted age, gender, nationality, English level, swiping familiarity, and the hand and finger used. They beat chance on every target. The same data that let a keyboard fit the hand also identify the person. The design conditions remain what they were in 2006, one hand, a moving body and a target measured in millimetres. What has changed is that the device now watches how it is held.

Sources

• Parhi, P., Karlson, A. K. and Bederson, B. (2006). Target size study for one-handed thumb use on small touchscreen devices. Mobile HCI. doi:10.1145/1152215.1152260 Two-phase study of one-handed thumb use, varying target size and screen position. Error rates changed little above about 9.6 mm for discrete targets and 7.7 mm for serial ones, so the authors recommended 9.2 mm and 9.6 mm.
• Henze, N., Rukzio, E. and Boll, S. C. (2011). 100,000,000 taps: analysis and improvement of touch performance in the large. Mobile HCI. doi:10.1145/2037373.2037395 A game in the Android Market recorded 120,626,225 touch events from 91,731 installations. Touch positions were systematically skewed, and a compensation function cut the error rate by 7.79 per cent in an independent test of 15,326,444 further touches.
• Bergstrom-Lehtovirta, J. and Oulasvirta, A. (2014). Modelling the functional area of the thumb on mobile touchscreen surfaces. International Conference on Human Factors in Computing Systems. doi:10.1145/2556288.2557354 Modelled the thumb's functional area from the kinematics of the gripping hand, fitted to trajectories from 20 participants. The model predicts the reachable area from surface size, hand size and index finger position on the back of the device.
• Azenkot, S. and Zhai, S. M. (2012). Touch behaviour with different postures on soft smartphone keyboards. Mobile HCI. doi:10.1145/2371574.2371612 Had 32 participants enter text on soft keyboards in three postures, two thumbs, one thumb and one index finger. Accuracy differed systematically between postures, with consistent offsets an algorithm could correct.
• Wobbrock, J., Myers, B. and Aung, H. (2008). The performance of hand postures in front- and back-of-device interaction for mobile computing. Int. J. Hum. Comput. Stud. doi:10.1016/j.ijhcs.2008.03.004 Three Fitts' law studies of postures across one and two hands, thumbs and index fingers, front and back of device. Two-handed postures beat one-handed ones at 1360 against 1575 milliseconds, index fingers beat thumbs at 1408 against 1527, and the back index finger beat the front thumb in horizontal movement but was worse at complex gestures.
• Goel, M., Findlater, L. and Wobbrock, J. (2012). WalkType: using accelerometer data to accomodate situational impairments in mobile touch screen text entry. International Conference on Human Factors in Computing Systems. doi:10.1145/2207676.2208662 Built WalkType, which reads the accelerometer to correct keystroke interpretation while walking, using data from 16 participants. Walking participants made 45.2 per cent fewer uncorrected errors and typed 12.9 per cent faster than in the control condition.
• Bohmer, M., Hecht, B. J., Schoning, J., Kruger, A. and Bauer, G. (2011). Falling asleep with Angry Birds, Facebook and Kindle: a large scale study on mobile application usage. Mobile HCI. doi:10.1145/2037373.2037383 Logged application usage from over 4,100 Android users. The average session lasted under a minute although users spent almost an hour a day on the phone, and the first application opened after waking was almost always a communication application.
• Oulasvirta, A., Rattenbury, T., Ma, L. Y. and Raita, E. (2011). Habits make smartphone use more pervasive. Personal and Ubiquitous Computing. doi:10.1007/s00779-011-0412-2 Described the checking habit, brief repeated inspection of quickly accessible dynamic content, across three studies. A field experiment showed checking reinforced by immediately available informational rewards, and habitual use experienced as an annoyance rather than an addiction.
• Palin, K., Feit, A., Kim, S., Kristensson, P. and Oulasvirta, A. (2019). How do People Type on Mobile Devices: Observations from a Study with 37,000 Volunteers. International Conference on Human-Computer Interaction with Mobile Devices and Services. doi:10.1145/3338286.3340120 In a web transcription task, 37,370 volunteers typed at a mean 36.2 words per minute with 2.3 per cent uncorrected errors. Auto-correct use correlated positively with entry rate and word prediction negatively.
• Dhakal, V., Feit, A., Kristensson, P. and Oulasvirta, A. (2018). Observations on Typing from 136 Million Keystrokes. International Conference on Human Factors in Computing Systems. doi:10.1145/3173574.3174220 Analysed 136 million keystrokes from 168,000 volunteers. Rollover typing was surprisingly prevalent, and clustering on inter-key intervals separated most users into eight groups differing in speed, accuracy and finger usage.
• Kane, S. K., Wobbrock, J. and Ladner, R. E. (2011). Usable gestures for blind people: understanding preference and performance. International Conference on Human Factors in Computing Systems. doi:10.1145/1978942.1979001 Ten blind and ten sighted people invented gestures and then performed a reference set. Blind participants preferred edge-based gestures and virtual key taps, and their gestures differed significantly in speed, size and shape.
• Hassan, S., Glasser, A., Shengelia, M., Starner, T., Forbes, S., Qualls, N. and Sepah, S. S. (2023). Tap to Sign: Towards using American Sign Language for Text Entry on Smartphones. Proceedings of the ACM on Human-Computer Interaction. doi:10.1145/3604274 Compared emulated fingerspelling recognition with a virtual keyboard for 12 Deaf participants. With practice, fingerspelling reached 42.5 words per minute against 31.9 for the keyboard. A second study preferred signing and the keyboard over fingerspelling for walk-up use.
• Lemarquis, D. C. A., Yilma, B. A. and Leiva, L. A. (2023). What can a swiped word tell us more Demographic and behavioural correlates from shape-writing text entry. Neural Computing and Applications. doi:10.1007/s00521-023-08559-0 Recurrent classifiers trained on swipe trajectories predicted age, gender, nationality, English level, swiping familiarity, and the hand and finger used, beating a random baseline on every target. Swiping therefore profiles users at a granularity the authors say raises privacy concerns.
• Karlson, A. K. and Bederson, B. (2007). ThumbSpace: Generalized One-Handed Input for Touchscreen-Based Mobile Devices. INTERACT. doi:10.1007/978-3-540-74796-3_30 Built ThumbSpace, which maps the whole screen onto a thumb-reachable region, and compared it with direct touch. Direct touch was faster, 811 milliseconds against 2068, while ThumbSpace was more accurate outside thumb reach, 94 per cent against 92, and most participants still preferred direct touch.