Human-AI teams: when does the combination beat the best of either?

Year 3 / Module 18 Human-AI interaction and machine learning interfaces / Unit 10 / Human-AI teams: when does the combination beat the best of either?

Human-AI teams: when does the combination beat the best of either?

HCI3102 · Unit 10 of 10

A human-AI combination outperforms the better of the two parties only when they make different errors and the delegation rule exploits that difference, a condition the meta-analytic evidence finds in a minority of studies, and the widely reported productivity gains come with convergence and deskilling costs that the headlines omit.

Human-AI teams: when does the combination beat the best of either?

Most AI product cases compare a person with the system against a person without it. This tenth of the ten units takes the combination, and the stricter question is whether it beats the better of the two parties working alone. Complementarity is a property to be measured, not a benefit that follows from putting a person and a model into one loop. It holds when the two parties make different errors, state confidence in comparable terms, and are combined by a rule that exploits the difference. The reported productivity gains are real, and they are largest for the least experienced. They are bought partly with the diversity of judgement that made a team worth having.

The combination is not the sum, and complementarity has conditions

Vaccaro, Almaatouq and Malone (2024) ran a pre-registered meta-analysis of 106 experimental studies reporting 370 effect sizes, each comparing humans, AI and a combination on one task. On average the combination performed significantly worse than the better of the two parties. The pooled effect was minus 0.23 (95 per cent confidence interval, minus 0.39 to minus 0.07). Combinations lost on decision tasks and gained significantly more on content-creation tasks. They gained where humans were already better and lost where the model was.

Bansal and colleagues (2021) studied three datasets with a model of human-comparable accuracy. Explanations did not increase the complementary gain from augmentation, and they raised the chance that a person accepted the recommendation, correct or not.

Steyvers and colleagues (2022) give the condition a form. Their Bayesian framework combines human and machine predictions with the different kinds of confidence each party expresses, and they applied it to a large image classification dataset. Complementarity held even when the two classifiers differed in accuracy, provided the difference fell inside a bound set by the latent correlation between their confidence signals.

Delegation is the rule, and advice is what a person does with it

Madras, Pitassi and Zemel (2017) formalise learning to defer, in which a model may pass the decision to a person. They show that accounting for the human's errors and the cost of deferring yields joint outcomes more accurate and less biased than rejection learning, which discards the case instead. Mozannar and Sontag (2020) supply the estimation machinery, learning a classifier and a rejector from samples of the expert's decisions alone. Their assumption is that the expert can be characterised, which a deployed product rarely manages. Wilder, Horvitz and Kamar (2020) train on the team's performance rather than the model's accuracy, so the machine takes instances that are hard for people. In scientific discovery and medical diagnosis, teams built this way beat machine and people separately.

Lai and Tan (2019) ran a deception-detection study varying machine assistance from human agency to automation. Explanations without a predicted label improved human performance slightly. Showing the label improved it by more than 20 per cent in relative terms. Assistance raised performance while lowering the person's own influence. Rastogi and colleagues (2022) formalise three cognitive biases in AI-assisted decisions and test a de-anchoring intervention and a time-allocation strategy that raised joint accuracy.

Tools that expose disagreement let a person act on it

Cai and colleagues (2019) built the interaction the argument implies. Working with pathologists, they identified what clinicians need when a retrieval algorithm's similarity does not match their diagnostic reasoning. They then built tools that let a user re-weight what matters. The tools raised diagnostic utility and trust, were preferred to a traditional interface, and cost no diagnostic accuracy. Gordon and colleagues (2022) show that the human side of a team is itself a decision. Their jury learning architecture models every annotator and classifies through a specified jury rather than a majority vote. Practitioners who used it built juries that altered 14 per cent of classification outcomes.

The productivity gains are real

Noy and Zhang (2023) ran a pre-registered experiment in which 453 college-educated professionals did incentivised writing tasks, half with ChatGPT. Time taken fell by 40 per cent, quality rose by 18 per cent, and inequality between workers fell. The task also assumes a live connection and a fast keyboard, so it says nothing about a metered link or a screen reader. Brynjolfsson, Li and Raymond (2023) studied a generative assistant introduced to 5,179 customer support agents. Issues resolved per hour rose 14 per cent on average, with 34 per cent for novice and low-skilled workers and little change for the experienced. Dell'Acqua and colleagues (2023) bound the claim. With 758 consultants, assistance on eighteen tasks inside the model's capability raised tasks completed by 12.2 per cent and cut time by 25.1 per cent.

The gains are bounded, and the cost the headline omits

On one task outside it the assisted group was 19 per cent less likely to be correct. Peng and colleagues (2023) recruited software developers to implement an HTTP server in JavaScript as quickly as possible, with a treatment group given access to GitHub Copilot. The treatment group finished 55.8 per cent faster than the control group. The observed effects varied with developer background, which the authors read as promise for people entering the field. The task is one small program in one language and the outcome is completion time, so the case measures a bounded coding exercise rather than a working week.

Fugener and colleagues (2021) combine an analytical model, experiments and a simulation to ask what advice does to the human side of the pair. Individual accuracy improved while unique human knowledge fell. Advised groups performed far worse than unassisted groups in a wisdom-of-crowds setting, because the convergence that helps each person destroys the diversity the group depended on. Personalising advice mitigated the loss. The gain and the loss come from the same study, so an adoption case that reports only the gain is incomplete.

Contested question: is complementarity designed or found?

The literature splits three ways. Vaccaro and colleagues (2024) and Bansal and colleagues (2021) read complementarity as a property of the task and of a person's ability to evaluate the output. On the pooled evidence the combination usually loses, and explanations do not rescue it. Steyvers and colleagues (2022), Wilder and colleagues (2020) and Madras and colleagues (2017) read it as a property of the composition. They hold that a joint model, a team objective and a deferral rule can manufacture the missing gains. Fugener and colleagues (2021) and Gordon and colleagues (2022) add a third position. The human in the loop is not fixed, so a rule calibrated on today's assistants may be tuned to a population the deployment erodes.

One pre-registered study would settle it. Fix the task, items, model and interface. Compare model alone, human alone, naive combination and a trained-to-complement combination. Report joint accuracy against the better party, the calibration of both confidence signals and the retained human knowledge.

What to do while the dispute is open

The positions are not equally evidenced. The claim that the combination usually loses rests on a pre-registered meta-analysis of 106 studies, the strongest design in this unit. The claim that complementarity can be manufactured rests on modelling papers and two applications, so it shows that a method works rather than how often. The claim that the human side moves is supported by one study that measures the loss directly. The strongest objection to this unit's thesis is that the pooled average may be the wrong unit of analysis. Vaccaro and colleagues (2024) report gains on content creation and on tasks where people were already better, so a rule that predicts when to combine may matter more than the average. While the dispute is open, a designer should state which party is expected to be better case by case, keep the two confidence signals comparable, and measure the retained diversity of human judgement alongside joint accuracy.

Where the argument is heading

The direction of travel is deferral trained on real expert behaviour and judged on joint outcomes. That is the programme Mozannar and Sontag (2020) and Wilder and colleagues (2020) set out. It joins two things. One is interfaces that make disagreement visible in the terms Cai and colleagues (2019) built. The other is an account of whose judgement is represented, which jury learning in Gordon and colleagues (2022) forces a team to write down.

The sources this unit rests on

Bansal, G., Wu, T., Zhou, J., et al. (2021). Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance. Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. doi:10.1145/3411764.3445717 Studied three datasets in which a model of human-comparable accuracy helped participants solve a task. Explanations did not increase the complementary gain from augmentation; they raised acceptance of the recommendation correct or not.
Brynjolfsson, E., Li, D. and Raymond, L. (2023). Generative AI at Work. NBER Working Paper 31161. doi:10.3386/w31161 Studied a generative assistant introduced to 5,179 customer support agents. Issues resolved per hour rose 14 per cent on average, with 34 per cent for novice and low-skilled workers and little change for the experienced.
Cai, C. J., Reif, E., Hegde, N., et al. (2019). Human-Centred Tools for Coping with Imperfect Algorithms During Medical Decision-Making. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. doi:10.1145/3290605.3300234 The published title uses the American spelling of the first term, rendered here in British form. They built tools that let pathologists re-weight what matters when a retrieval algorithm's similarity does not match their reasoning; the tools raised diagnostic utility and trust without loss of accuracy.
Dell'Acqua, F., McFowland, E., III, Mollick, E. R., et al. (2023). Navigating the Jagged Technological Frontier. Harvard Business School Working Paper 24-013. doi:10.2139/ssrn.4573321 A pre-registered field experiment with 758 consultants. On eighteen tasks inside the model's capability, assistance raised tasks completed by 12.2 per cent and cut time by 25.1 per cent; on one task outside it, the assisted group was 19 per cent less likely to be correct.
Fugener, A., Grahl, J., Gupta, A., et al. (2021). Will Humans-in-the-Loop Become Borgs? Merits and Pitfalls of Working with AI. MIS Quarterly 45(3), 1527-1556. doi:10.25300/misq/2021/16553 An analytical model, experiments and a simulation on what advice does to complementarity. Individual accuracy improved while unique human knowledge fell, and advised groups performed far worse than unassisted groups in a wisdom-of-crowds setting; personalising advice mitigated the loss.
Gordon, M. L., Lam, M. S., Park, J. S., et al. (2022). Jury Learning: Integrating Dissenting Voices into Machine Learning Models. CHI Conference on Human Factors in Computing Systems. doi:10.1145/3491102.3502004 Introduced jury learning, which models every annotator and classifies through a specified jury rather than a majority vote. Practitioners built juries that altered 14 per cent of outcomes, so whose judgement a system embodies is a design decision.
Lai, V. and Tan, C. (2019). On Human Predictions with Explanations and Predictions of Machine Learning Models. Proceedings of the Conference on Fairness, Accountability, and Transparency. doi:10.1145/3287560.3287590 A deception-detection study varying machine assistance from human agency to automation. Explanations alone improved performance slightly, predicted labels improved it relatively by 20 per cent or more, and explanations induced accuracy close to an explicit statement of machine strength.
Madras, D., Pitassi, T. and Zemel, R. (2017). Predict Responsibly: Improving Fairness and Accuracy by Learning to Defer. arXiv preprint arXiv:1711.06664. doi:10.48550/arXiv.1711.06664 Formalised learning to defer, in which a model predicts or passes the case to a person. Accounting for the human's errors and the cost of deferring gives joint outcomes more accurate and less biased than rejection learning, even with a biased user.
Mozannar, H. and Sontag, D. (2020). Consistent Estimators for Learning to Defer to an Expert. Proceedings of the 37th International Conference on Machine Learning, PMLR 119, 7076-7087. https://proceedings.mlr.press/v119/mozannar20b.html Gives a procedure for a predictor that either classifies or defers to a downstream expert, learned from samples of the expert's decisions with a consistent surrogate loss. It needs a characterisation of the expert that a deployed product rarely has.
Noy, S. and Zhang, W. (2023). Experimental Evidence on the Productivity Effects of Generative Artificial Intelligence. Science 381(6654), 187-192. doi:10.1126/science.adh2586 A pre-registered experiment in which 453 college-educated professionals did incentivised writing tasks, half with ChatGPT. Time taken fell 40 per cent, quality rose 18 per cent and inequality fell, and the authors read the tool as substituting for effort rather than complementing skill.
Peng, S., Kalliamvakou, E., Cihon, P., et al. (2023). The Impact of AI on Developer Productivity: Evidence from GitHub Copilot. arXiv preprint arXiv:2302.06590. doi:10.48550/arXiv.2302.06590 Developers were recruited to implement an HTTP server in JavaScript as quickly as possible, with a treatment group given GitHub Copilot. The treatment group finished 55.8 per cent faster, and the observed heterogeneity suggests the tool helps people entering software development.
Rastogi, C., Zhang, Y., Wei, D., et al. (2022). Deciding Fast and Slow: The Role of Cognitive Biases in AI-assisted Decision-making. Proceedings of the ACM on Human-Computer Interaction 6(CSCW1). doi:10.1145/3512930 Model three cognitive biases in AI-assisted decisions and test interventions against them. A de-anchoring strategy improved joint accuracy, and a time-allocation strategy with explanation reached better collaboration under assumed conditions.
Steyvers, M., Tejeda, H., Kerrigan, G., et al. (2022). Bayesian modelling of human-AI complementarity. Proceedings of the National Academy of Sciences 119(11). doi:10.1073/pnas.2111547119 The published title uses the American spelling of the third word and an en dash; both are rendered here in house form. A Bayesian framework combines human and machine predictions with the confidence each expresses; complementarity held across accuracy differences inside a bound set by the latent correlation of their confidence signals.
Vaccaro, M., Almaatouq, A. and Malone, T. (2024). When combinations of humans and AI are useful: A systematic review and meta-analysis. Nature Human Behaviour. doi:10.1038/s41562-024-02024-1 A pre-registered systematic review and meta-analysis of 106 experimental studies reporting 370 effect sizes, each comparing humans, AI and a combination. Combinations were on average worse than the better party; gains came on content-creation tasks and where humans were better, losses on decision tasks.
Wilder, B., Horvitz, E. and Kamar, E. (2020). Learning to Complement Humans. Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, 1526-1533. doi:10.24963/ijcai.2020/212 Trained models end to end on the performance of the human-machine team rather than the model's own accuracy, so the model takes instances that are hard for people. In scientific discovery and medical diagnosis the resulting teams beat machines and people separately.

Check your understanding & apply it · 4 questions, 1 essay

1A vendor reports that in its study the model plus a person beat the person working alone. Diagnose what that result establishes about complementarity.

2A team pairs a strong model with a clinician, has both parties state confidence on comparable scales, and combines their predictions with a simple average. What does Steyvers and colleagues (2022) predict about when that combination gains?

3A vendor asks to generalise a cut of 40 per cent in the time taken from a writing experiment to a firm's whole workforce. How does the unit's evidence judge that request?

4A firm adopts AI advice for its forecasters and finds each individual more accurate while the group's aggregate forecasts become worse. Diagnose the outcome with the unit's evidence.

5Essay A radiology department proposes an AI triage tool for chest imaging. The human-in-the-loop design shows the model's suggested finding above the image with a confirm button, and the pilot reports three accuracy figures on a held-out sample of 400 cases: model alone 0.88, radiologists alone 0.85, radiologists using the tool 0.86. The vendor's summary states that the pilot shows complementary human-AI performance, and the department asks you to review the proposal before procurement. Write the review. Say whether the three figures establish complementarity and what they do not show, name the mechanism the design omits, propose the changes that would give the combination a chance of exceeding the better party, specify what you would measure in the next evaluation, and state which result would make you recommend against deployment.

The figures do not establish complementarity, because the comparison that matters is missing. A combination is complementary only when it beats the better of the two parties working alone (Vaccaro, Almaatouq and Malone, 2024), and here the better party is the model at 0.88 while the combination reaches 0.86. The pilot therefore shows a small loss against the strongest arm, which is the pattern the meta-analysis reports on average. What the figures cannot show is why. The design shows the model's suggestion first and asks for confirmation, which is the arrangement Lai and Tan (2019) found raises accuracy while transferring influence from the person to the machine, and Bansal and colleagues (2021) found raises acceptance of a recommendation whether or not it is correct. Confirmation after the answer is visible is assent, not a second independent judgement, so the two error structures are unlikely to differ in the way complementarity needs (Steyvers and colleagues, 2022). The design change is to obtain the radiologist's own read before the model is shown, keep the model's confidence available, and route cases on a stated rule, which is what learning to defer does when it accounts for the human's errors and the cost of deferring (Madras, Pitassi and Zemel, 2017) and what training on the team's outcome rather than the model's accuracy is for (Wilder, Horvitz and Kamar, 2020). The next evaluation should be pre-registered, fix the case set and the interface, and compare four arms: model alone, radiologist alone, the current design and the independent-read design. It should report joint accuracy against the better party with a confidence interval, the calibration of both parties' confidence signals, and a measure of retained human knowledge, following the convergence result of Fugener and colleagues (2021). Watch the case mix too, since assistance helped on eighteen tasks inside the model's capability and harmed on one outside it in Dell'Acqua and colleagues (2023). I would recommend against deployment if the independent-read arm still fails to exceed the model alone on a powered sample, or if the model's errors are concentrated on the rare findings where the radiologist's judgement is the only defence.