Menu structure is a cost model, and breadth beats depth only under the assumptions the studies make explicit

Year 2 / Module 15 Interaction techniques and usability evaluation / Unit 3 / Menu structure is a cost model, and breadth beats depth only under the assumptions the studies make explicit

Menu structure is a cost model, and breadth beats depth only under the assumptions the studies make explicit

HCI2107 · Unit 3 of 10

The breadth-over-depth result is a prediction derived from an assumed cost per decision and per movement, so it holds for reaching a known item with stable labels and dissolves when label quality, learning and navigation dominate; treating a derived cost function as a design rule mistakes one model of the task for the structure of the task itself.

Menu structure is a cost model, and breadth beats depth only under the assumptions the studies make explicit

Every menu is a claim about cost. A panel of eight items rather than three bets that the decisions are cheap and the extra pointing is expensive. Burying the same eight behind two submenus bets the other way. The rule of thumb that settles the question has an origin. Breadth beats depth, this third of ten units argues, is not a finding about menus. It is a prediction derived from an assumed cost per decision and per movement, and it holds only for reaching a known item under stable labels.

Miller (1956) is usually cited to license the rule. He reviewed the evidence on the span of absolute judgement and of immediate memory, reporting a span of about seven items, stretched by chunking. That is a statement about how much a person holds. It is not a statement about how many items belong on a screen.

The empirical case for breadth carried its own caveat

Landauer and Nachbar (1985) gave the result its footing. Participants selected goal items from touch menus while the number of alternatives per step was varied, which changed the target area inversely. Mean response time per screen was well described by a function of the logarithm of the number of alternatives. The function has a decision component in series with a movement component, in agreement with the Hick-Hyman and Fitts laws. On that fitted function breadth beats depth. The authors state the caveat the rule forgets: this function favours breadth, whereas others might not.

Snowberry, Parkinson and Sisson (1983) ran the structural comparison early. They arranged 64 words as six levels of two choices, three of four, two of eight, or one menu of 64. The trend analysis rested on the broad menu that kept each category's words together rather than the one placing them at random, so the advantage arrived with the grouping as well as the breadth.

The optimum is derived, and one assumption moves it

Lee and Macgregor (1985) turned the intuition into an optimisation. They derived the number of alternatives per page that minimises search time from search strategy, scanning time, key-press time and computer response time. They reported an optimum of four to eight, with limiting values of three and four for exhaustive and self-terminating search. The optimum depends on key-press and system response time, so a faster system moves it, and a slow or metered connection pays the same term. The right breadth is an output of a cost model containing terms that belong to the machine.

Paap and Roske-Hofstrand (1986) reran the derivation under one changed assumption. The earlier analysis, they note, applies only where the user cannot restrict the search by experience with the panel or knowledge of the grouping. Allow restricted search under optimal grouping, and the optimal number of alternatives per page ranges from 16 to 78, roughly an order of magnitude away. Navigation to a desired location also grows harder with depth, a cost the function omits.

Norman and Chin (1988) tested tree shape rather than deriving it. They grouped 256 gift items into four-level trees of five shapes, and they varied whether the target was named. For named targets, search time was about the same across all five structures. For scenario targets the structures differed greatly. A scenario target is one the user seeks as appropriate to a situation rather than by name, and there the concave and increasing shapes were faster than the constant one while the convex and decreasing shapes were slower. Where the breadth sits matters, and whether the target can be named decides whether the effect appears.

Structure is not the only variable in the cost

Callahan, Hopkins, Weiser and Shneiderman (1988) compared pie menus with linear menus. They held the commands, the item count and the depth constant, and changed only the geometry. Pie menus reduce target seek time and lower error rates, because they fix the distance factor and enlarge the target under Fitts's law. Against linear menus they were subjectively equivalent.

Sears and Shneiderman (1994) changed neither breadth nor geometry but the order of the items. Split menus place high-frequency items at the top, and performance times fell by 17 to 58 per cent across two in situ usability studies. A controlled experiment found them significantly faster than alphabetic menus, and participants rated them significantly higher. The gain depends on the frequency distribution being stable and known, so the method presupposes a task that repeats.

Uddin, Gutwin and Lafreniere (2016) used the user's own hand as a landmark on a large multi-touch surface. They built menus of 42 and 160 commands and compared them with tab widgets. The small version was significantly faster at all stages. The large version was slower at first and equivalent to tabs after experience, with no difference in error rates. Participants strongly preferred both.

No single measure decides a menu, and a stated model predicts it

The expert poses the sharper problem. Kurtenbach and Buxton (1993) studied hierarchic marking menus. In these, a skilled user selects an item by drawing a mark rather than pressing anything. The authors measured how many items a level can carry, and how deep the hierarchy can go before a mark becomes too slow or too error-prone. Error rates stayed below ten per cent for menus of breadth eight at depth two, and for four-item menus even four levels deep. Beyond breadth eight at depth greater than two, selection became error-prone even for an expert. Where it worked, breadth and depth traded evenly. Reaching 64 items through four-item menus three levels deep was about as fast as eight-item menus two levels deep, with equivalent error rates.

Findlater and McGrenere (2004) show that speed and preference can point apart. They ran a controlled study with 27 participants comparing static, adaptable and adaptive split menus. The static menu was significantly faster than the adaptive one. The adaptable menu was faster than the adaptive one under certain conditions. Most participants preferred the adaptable menu.

Cockburn, Gutwin and Greenberg (2007) assembled the components into a predictive model of menu performance. It combines Fitts's law pointing time, visual search time while the user is a novice, Hick-Hyman decision time once the user is expert, and a term for the transition between the two. They tested it against traditional, split and adaptive morphing menus, and the data matched the predictions extremely well.

Contested question: which structure is right, and what evidence would settle it?

The literature disagrees on three connected questions, and the disagreements turn on assumptions as much as on data.

The first is what the optimum breadth is. Lee and Macgregor (1985) place it at four to eight alternatives per page. Paap and Roske-Hofstrand (1986) place it at 16 to 78 by allowing restricted search. Both figures come from the same style of cost function, so the disagreement is about whether search is exhaustive or restricted. An experiment holding the item set and task fixed, while manipulating whether the participant can restrict the search, would settle it.

The second is whether structure is the right variable at all. Landauer and Nachbar (1985) measure time against breadth and depth. Callahan and colleagues (1988), Sears and Shneiderman (1994) and Uddin and colleagues (2016) change time and error without changing either. A factorial study that varies breadth, depth and geometry independently under a fixed command set and device would settle it, and it would report error and learning as well as time. No such study appears in this unit's list.

The third is which measurement decides a menu. Kurtenbach and Buxton (1993) find error the binding constraint for experts, and Findlater and McGrenere (2004) find preference pointing away from speed. A design is under-specified by any single number, so what settles the question is which measurement the claim is about. A study reporting time, error, learning and preference for the same two structures would show how often those measures agree.

The evidence is mixed in kind. Lee and Macgregor (1985) and Paap and Roske-Hofstrand (1986) derive optima from cost functions rather than measure them, Norman and Chin (1988) test tree shape in a controlled study, and Cockburn, Gutwin and Greenberg (2007) fit one model to three menu types. The strongest objection is that the unit's claim is unfalsifiable, because a cost model with four terms fits almost any menu result. A designer should state the assumptions and report error and learning beside time while the dispute is open.

Where the argument is heading

Recent work states the cost model and optimises against it, with a price for the disruption that change itself causes. Todi, Bailly, Leiva and Oulasvirta (2021) adapt a menu with model-based reinforcement learning that consults predictive HCI models. It accounts for the costs of surprise and relearning, and it changes when that helps and not when it does not. In the adaptive menu case it outperformed a non-adaptive and a frequency-based policy. Uddin and colleagues (2016) use the same stance to choose a new geometry. Both depend on the model of Cockburn and colleagues (2007), which is why its assumptions are now the practical question. An optimiser delivers whatever it was told to minimise, so a model pricing only pointing and decision time produces an interface that is quick for an expert and unexplained for a novice.

The sources this unit rests on

Callahan, J., Hopkins, D., Weiser, M. and Shneiderman, B. (1988). An empirical comparison of pie vs. linear menus. Proceedings of the SIGCHI conference on Human factors in computing systems - CHI '88. doi:10.1145/57167.57182 Found pie menus reduce target seek time and lower error rates by fixing the distance factor and enlarging the target under Fitts's law, while being subjectively equivalent. Geometry, not only structure, sets a menu's cost.
Cockburn, A., Gutwin, C. and Greenberg, S. (2007). A predictive model of menu performance. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. doi:10.1145/1240624.1240723 Combined pointing, novice visual search, expert decision time and the novice-to-expert transition into one model, validated against traditional, split and adaptive menus. Recent optimisers consult it.
Findlater, L. and McGrenere, J. (2004). A comparison of static, adaptive, and adaptable menus. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. doi:10.1145/985692.985704 Found the static menu significantly faster than the adaptive one with 27 participants, while most preferred the adaptable menu. Performance and preference diverge.
Kurtenbach, G. and Buxton, W. (1993). The limits of expert performance using hierarchic marking menus. Proceedings of the SIGCHI conference on Human factors in computing systems - CHI '93. doi:10.1145/169059.169426 Measured the breadth and depth limits of expert marking menus. Error rates stayed below ten per cent to breadth eight at depth two, and 64 items through four-item menus three levels deep was about as fast as eight-item menus two levels deep.
Landauer, T. K. and Nachbar, D. W. (1985). Selection from alphabetic and numeric menu trees using a touch screen. Proceedings of the SIGCHI conference on Human factors in computing systems - CHI '85. doi:10.1145/317456.317470 Varied the number of alternatives per screen and found mean response time well described by a decision component in series with a movement component, against the Hick-Hyman and Fitts laws. That function favours breadth, and the authors state that others might not.
Lee, E. and Macgregor, J. (1985). Minimizing user search time in menu retrieval systems. Human Factors. doi:10.1177/001872088502700203 Derived the number of alternatives per page that minimises search time from scanning, key-press and computer response time. The optimum is four to eight, with limiting values of three and four for exhaustive and self-terminating search.
Miller, G. A. (1956). The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychological Review. doi:10.1037/h0043158 Reviewed the evidence that the span of absolute judgement and of immediate memory impose severe limits, reporting a span of about seven items, stretched by chunking. It is cited to license a menu rule, yet it describes a memory span.
Norman, K. L. and Chin, J. P. (1988). The effect of tree structure on search in a hierarchical menu selection system. Behaviour and Information Technology. doi:10.1080/01449298808901862 Grouped 256 gift items into four-level trees of five shapes and varied whether the target was named. Search time barely differed for named targets but differed greatly for scenario targets, the concave and increasing shapes faster than the constant one.
Paap, K. R. and Roske-Hofstrand, R. J. (1986). The optimal number of menu options per panel. Human Factors. doi:10.1177/001872088602800401 Extended the Lee and Macgregor analysis to restricted search, where optimal grouping places the optimal number of alternatives per page at 16 to 78 and navigation difficulty grows with depth.
Sears, A. and Shneiderman, B. (1994). Split menus. ACM Transactions on Computer-Human Interaction. doi:10.1145/174630.174632 Placed high-frequency items at the top of menus and reduced performance times by 17 to 58 per cent across two in situ usability studies, a controlled experiment showing split menus significantly faster than alphabetic menus. The benefit depends on a stable, known frequency distribution.
Snowberry, K., Parkinson, S. R. and Sisson, N. (1983). Computer display menus. Ergonomics. doi:10.1080/00140138308963390 Arranged 64 words as six levels of two choices, three of four, two of eight, or one menu of 64, and found search speed and accuracy improved with breadth. The trend analysis used the broad menu that kept each category's words together, so the advantage arrived with grouped labels.
Todi, K., Bailly, G., Leiva, L. and Oulasvirta, A. (2021). Adapting user interfaces with model-based reinforcement learning. Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. doi:10.1145/3411764.3445497 Adapted interfaces with model-based reinforcement learning that accounts for the costs of surprise and relearning and consults predictive HCI models. In the adaptive menu case it outperformed both a non-adaptive and a frequency-based policy.
Uddin, M. S., Gutwin, C. and Lafreniere, B. (2016). HandMark menus: rapid command selection with the non-dominant hand. Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. doi:10.1145/2858036.2858211 Built hand-centred menus of 42 and 160 commands using the hand as a landmark on a multi-touch surface, comparing them with tab widgets. The small version was significantly faster at all stages; the large version was slower at first but equivalent to tabs after experience.

Check your understanding & apply it · 4 questions, 1 essay

1A team designs a menu for users who know which category a command sits in but not its label, and who search each panel exhaustively because experience cannot narrow the search. What does the unit predict for the optimum number of items per panel?

2One team places its ten commonest commands at the top of a menu and reports a large time saving in its own product. A second team copies the layout for a product whose usage is evenly spread and finds no gain. Which diagnosis fits the unit?

3A team must choose between a structure that is faster for experts and one that users prefer. Which judgement does the unit support?

4A student fits a four-term cost model to a menu study after the data were collected, obtains a close match and reports that the structure's cost is confirmed. Which diagnosis matches the unit's own strongest objection to its argument?

5Essay A product team is rebuilding a settings area with 120 options. It proposes two structures: a broad one, a first panel of twelve items each opening a panel of about ten, and a deep one, three levels of about five items each. An expert user reports that the broad version is slow to scan, and a novice reports that the deep version loses them. You have been asked to design the evaluation that would settle which structure to ship. Design it. State what each contested position in this unit's literature predicts for the two structures, name the sources you rely on, say what you would measure beyond selection time, and state what result would count for breadth, for depth, and for the claim that breadth and depth are the wrong variables.

Begin by naming the two structures in the terms the literature uses, because the argument is about a cost assumption rather than about taste. The breadth prediction is Landauer and Nachbar (1985), whose fitted function of the logarithm of the number of alternatives, a decision component in series with a movement component, favours breadth over depth for a known target. It predicts the deep structure, with its extra decisions but shorter movements, to be slower. The counter-derivation is Lee and Macgregor (1985), whose optimum of four to eight alternatives per page assumes an exhaustive search, against Paap and Roske-Hofstrand (1986), who allow restricted search and obtain 16 to 78 across the same parameter values. Since the broad first panel of twelve sits inside both ranges and the deep first panel of five sits inside one and outside the other, the study has to fix which search model is assumed rather than let the team pick the figure that flatters its design. Run a within-subjects comparison of the two structures on the same 120 commands and the same tasks, and manipulate the search model as a factor: half the tasks name the setting exactly, half describe a goal the user must interpret, following Norman and Chin (1988), who found the effect of tree shape appeared for scenario targets and largely vanished for named ones. Measure more than time. Record navigation steps and error rate, because Kurtenbach and Buxton (1993) found error was the binding constraint for experts and that breadth and depth traded evenly once it was respected; record preference and perceived control, because Findlater and McGrenere (2004) found the static menu faster than the adaptive one while most users preferred the one they controlled; and record search time at first contact and again after repeated use, because Uddin and others (2016) found a large technique slower at the start but equivalent after experience, so a single measurement point would have reversed the recommendation. Hold geometry constant, since Callahan and colleagues (1988) changed only layout and changed both time and error. A breadth result requires the broad structure to be faster or less error-prone for named targets without losing on scenario targets or on early-use time. A depth result requires the reverse. A result for the third position requires the two structures to be indistinguishable once the named and scenario tasks are separated, while item order or geometry produces a difference the structures do not, which would show that the breadth-depth axis was never the operative variable here.