Who decides? Mixed initiative and the allocation of control

This first of the ten units takes the question of who decides. An AI feature can be a hint the user ignores, an interruption the user answers, or an action the system takes alone. The design literature calls that difference a question of autonomy. Horvitz (1999) set out an alternative. An automated service should act when its expected benefit exceeds the expected cost of acting wrongly. Two decades of principles and guidelines have supplied the vocabulary for that comparison. With few exceptions, they have supplied no measurement of whether an allocation improves what person and system achieve together.

The question is not how autonomous the system is

Horvitz's (1999) principles for mixed-initiative interfaces are the origin of much of what the field now does. They ask the designer to provide automation that adds genuine value over direct manipulation. They ask the designer to infer the best action from its costs and benefits, and to make a poor guess cheap. The decision to act was therefore a calculation about consequences, not a property of the model.

Heer (2019) argues from the design side that the choice between full automation and full human control is a false one. That is an agenda as much as a finding, so it supplies the framing rather than an answer.

Mixed initiative was named, and its measurement left open

Allen, Guinn and Horvitz (1999) offered a taxonomy of dialogue problems, an account of uncertainty and Bayesian assistance, and a closing essay that confronted evaluation directly. That essay treats measurement as open, so the gap the unit describes is a question left unclosed.

Most of the work cited here studies one side of the pair. One measurement of the other side comes from end-user debugging. Kulesza and colleagues (2010) ran a formative study of how people reason about and correct a text-classification program's output. They then built and evaluated an end-user debugging prototype with 74 participants. Users given the program's runtime information improved the learned program's accuracy by a mean of 18 per cent. Users given the logic explanations, alone or with runtime, did not improve it. The outcome measured was the learned program's accuracy on one coding task, not trust and not joint performance.

The cost of a wrong act falls on a named party

The comparison is awkward because its two costs fall on different people. A false alarm costs the operator time and attention. A missed detection costs whoever the system was meant to protect. The weights belong to the parties who bear the errors. That is Horvitz's (1999) criterion: expected utility, including the cost of interrupting.

Avrahami, Fogarty and Hudson (2007) compared self-reports of people's own interruptibility at work with a second group's estimates of the same moments from recordings. A closed office door changed observers' estimates without changing participants' own reports. The input a system would most naturally request is therefore unreliable.

Delegation is a graded judgement, not a property of the task

Lubars and Tan (2019) built a framework of task delegability from motivation, difficulty, risk and trust, and surveyed willingness to delegate 100 tasks. They found little preference for full AI control and a strong preference for machine-in-the-loop designs led by the person.

Who proposes the handover changes its cost. Adam and colleagues (2024) ran two online experiments in a software development setting. They compared delegation the user initiates with delegation the system initiates. System-invoked delegation raised perceived self-threat and lowered willingness to accept.

Model-initiated delegation improved the pair in one experiment

The strongest measurements of the pair here let the model decide when to hand over. Hemmer and colleagues (2023) ran an experiment with 196 participants. A model trained to decide whether to predict an instance or delegate it did so. The authors measured task performance, satisfaction and self-efficacy. Performance and satisfaction improved whether or not participants knew the delegation came from the model.

Pinski, Adam and Benlian (2023) add the sharper result. In a between-subjects experiment with 111 participants, people taught about AI delegated more in line with their task appraisal and performed better.

Expectations decide how a stated error rate is read

Because an AI feature is wrong sometimes, people judge its errors against what they expected. Kocielnik, Amershi and Bennett (2019) used a scheduling assistant with a stated accuracy of 50 per cent. They varied which error it was designed to avoid. They also varied how its limits were set before first use. The same accuracy produced different perceptions and acceptance, so a stated error rate is a design variable.

Tullio and colleagues (2007) tracked eight office workers for six weeks as they used a system predicting their managers' interruptibility. Participants discounted some initial misconceptions while the structure of their models stayed stable.

Guidelines supply the vocabulary, not the measurements

The most cited codification here is a set of guidelines, not a set of findings. Amershi and colleagues (2019) distilled 18 design guidelines for human-AI interaction. They validated the guidelines with 49 design practitioners who applied them to 20 AI-infused products. The validation established relevance across scenarios, not whether following the guidelines improves any outcome.

The genre has moved to generative systems without acquiring outcome evidence. Weisz and colleagues (2024) developed six design principles for generative AI applications, each paired with implementable strategies. Shneiderman (2020) makes the strongest claim in this style. He asserts that high human control and high computer automation can be designed together, without comparative evidence.

Contested question: who should hold the initiative, and does any allocation improve the joint outcome?

Three positions are live, and they are not a preference for more or less automation.

The first holds that the person should initiate the handover. Adam and colleagues (2024) found that system-invoked delegation raised self-threat and lowered acceptance. Lubars and Tan (2019) found a strong preference for designs in which the person leads, and Shneiderman (2020) keeps human control high.

The second holds that the system may take the initiative when that improves the pair. Hemmer and colleagues (2023) found that AI-initiated delegation improved performance and satisfaction in 196 participants. Heer (2019) treats interleaving automation with judgement as the aim rather than a concession.

The third holds that the dispute cannot be settled yet, because the studies collected here have not measured the comparison the field recommends. The guidelines literature (Amershi et al., 2019; Weisz et al., 2024) and Shneiderman's (2020) framework supply the vocabulary without a comparative measurement of joint performance.

What would settle it is a design that varies the initiator while holding the interface constant. It would measure joint performance and the human outcome of acceptance, satisfaction and self-efficacy in the same participants. Pinski, Adam and Benlian (2023) show why one score cannot answer it. Their better delegation improved performance and reduced intended adoption in the same experiment.

What to do while the dispute is open

The two camps do not carry equal evidence, and neither carries the evidence the dispute needs. That the person should initiate rests on two online experiments and one survey of willingness, which measure a psychological cost rather than joint performance. That the system may take the initiative rests on a single randomised experiment with 196 participants. Neither side reports the error rate of the combined pair. The strongest objection to this unit's thesis is that the joint outcome is the wrong objective. Pinski, Adam and Benlian (2023) found that better-informed delegation improved performance and lowered the intention to use the system, so an allocation that wins on the pair can still lose the person. While the dispute is open, a designer should make the system's initiative visible, reversible and cheap to decline, a prompt a screen reader can read, a keyboard can answer and a shared device can receive, and record who initiated each handover beside what the pair achieved.

Where the argument is heading

The direction of travel is from guidance towards stated policy. Weisz and colleagues (2024) pair each principle with implementable strategies, moving the advice towards something a team can audit against. The measurement the area still lacks is an allocation rule stated as policy and tested on the joint outcome with the human outcome beside it. Pinski, Adam and Benlian (2023) do that for one variable. Until then, a claim that a feature improves the pair remains a design argument.

The sources this unit rests on

• Horvitz, E. (1999). Principles of mixed-initiative user interfaces. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 159-166. doi:10.1145/302979.303030 Horvitz sets out principles for coupling automated services with direct manipulation, among them to weigh expected costs and benefits before acting and to make a poor guess cheap. The Lookout assistant chose between silence, a question and an action by expected utility, so the founding statement is a rationale, not a measurement.
• Allen, J. E., Guinn, C. I. and Horvitz, E. (1999). Mixed-initiative interaction. IEEE Intelligent Systems 14(5), 14-23. doi:10.1109/5254.796083 Three essays established the area: a taxonomy of dialogue problems, an account of uncertainty and Bayesian assistance, and a closing essay on evaluating such systems, including test sets and metrics for dialogue models. That essay treats measurement as open, so the unit reads its gap as a question left unclosed.
• Heer, J. (2019). Agency plus automation: Designing artificial intelligence into interactive systems. Proceedings of the National Academy of Sciences 116(6), 1844-1850. doi:10.1073/pnas.1807184115 Heer argues that the choice between full automation and full human control is a false one and describes systems in which computational assistance augments rather than replaces judgement. It is a design argument and an agenda, supplying the unit's framing without answering which allocation performs better.
• Avrahami, D., Fogarty, J. and Hudson, S. E. (2007). Biases in human estimation of interruptibility. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 50-60. doi:10.1145/1240624.1240632 The authors compared self-reports of people's own interruptibility at work with a second group's estimates of the same moments from recordings. A closed office door changed observers' estimates without changing participants' own reports, so the input a system would most naturally request is unreliable.
• Lubars, B. and Tan, C. (2019). Ask not what AI can do, but what AI should do: Towards a framework of task delegability. arXiv preprint arXiv:1902.03245. doi:10.48550/arXiv.1902.03245 The authors built a framework of task delegability from motivation, difficulty, risk and trust and surveyed willingness to delegate 100 tasks drawn from papers, media and everyday life. They found little preference for full AI control and a strong preference for machine-in-the-loop designs led by the person, with trust most correlated with the preferred arrangement.
• Adam, M., Diebel, C., Goutier, M. and Benlian, A. (2024). Navigating autonomy and control in human-AI delegation: User responses to technology- versus user-invoked task allocation. Decision Support Systems 180, 114193. doi:10.1016/j.dss.2024.114193 In two online experiments in a software development setting, the authors compared user-invoked with system-invoked delegation. System-invoked delegation raised perceived self-threat and lowered willingness to accept, so the cost of an allocation depends on which party proposes it.
• Hemmer, P., Westphal, M., Schemmer, M., et al. (2023). Human-AI collaboration: The effect of AI delegation on human task performance and task satisfaction. Proceedings of the 28th International Conference on Intelligent User Interfaces, 453-463. doi:10.1145/3581641.3584052 In an experiment with 196 participants, a model trained to decide whether to predict or delegate did so, and the authors measured task performance, satisfaction and self-efficacy. Performance and satisfaction improved whether or not participants knew the delegation came from the model.
• Pinski, M., Adam, M. and Benlian, A. (2023). AI knowledge: Improving AI delegation through human enablement. Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, 1-17. doi:10.1145/3544548.3580794 A between-subjects experiment with 111 participants tested whether teaching people about AI improves how they delegate. Participants given AI knowledge aligned their delegation with their task appraisal and performed better, but the same knowledge lowered their intention to use AI.
• Kocielnik, R., Amershi, S. and Bennett, P. N. (2019). Will you accept an imperfect AI? Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, 1-14. doi:10.1145/3290605.3300641 Using a scheduling assistant with a stated accuracy of 50 per cent, the authors varied which error it was designed to avoid and how its limits were set before first use. The same accuracy produced different perceptions and acceptance, so a stated error rate is a design variable.
• Tullio, J., Dey, A. K., Chalecki, J. and Fogarty, J. (2007). How it works: A field study of non-technical users interacting with an intelligent system. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 31-40. doi:10.1145/1240624.1240630 In a six-week field study, the authors interviewed eight office workers about a system predicting their managers' interruptibility. Participants discounted some initial misconceptions while the structure of their models stayed stable.
• Kulesza, T., Stumpf, S., Burnett, M., et al. (2010). Explanatory Debugging: Supporting End-User Debugging of Machine-Learned Programs. 2010 IEEE Symposium on Visual Languages and Human-Centric Computing. doi:10.1109/vlhcc.2010.15 A formative study of how users reason about and correct a text-classification program's output led to an end-user debugging prototype evaluated with 74 participants. Users given the program's runtime information improved the learned program's accuracy, while the logic views did not, so the measured change is the program's accuracy.
• Amershi, S., Weld, D., Vorvoreanu, M., et al. (2019). Guidelines for human-AI interaction. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, 1-13. doi:10.1145/3290605.3300233 The authors distilled 18 generally applicable design guidelines and validated them with 49 design practitioners who applied them to 20 popular AI-infused products. The validation established relevance across scenarios, not whether following the guidelines improves any outcome.
• Weisz, J. D., He, J., Muller, M., et al. (2024). Design principles for generative AI applications. Proceedings of the CHI Conference on Human Factors in Computing Systems, 1-22. doi:10.1145/3613904.3642466 The authors developed six design principles for generative AI applications, each paired with implementable strategies, through literature review, practitioner feedback and use in two applications. They remain practitioner guidance developed with experts rather than tested interventions.
• Shneiderman, B. (2020). Human-centred artificial intelligence: Reliable, safe and trustworthy. International Journal of Human-Computer Interaction 36(6), 495-504. doi:10.1080/10447318.2020.1741118 Shneiderman proposes a two-dimensional framework separating the level of human control from the level of computer automation and argues that high levels of both can be designed together. The combination is asserted as desirable without comparative evidence.