Who fails when a control fails: warnings, defaults and the economics of security

A control fails when someone does not use it, and the usual reading is that the person failed. This tenth and last unit argues that this names the wrong party: the evidence points at the control as the tractable fault. Where the safe action can be made the default, remove the prompt; where it cannot, measure the burden rather than assert it.

The failure is located before it is explained

Anderson (2001) located the failure economically. Information insecurity, he argues, is at least as much a matter of perverse incentives as of weak technology, analysed through network externalities, moral hazard and adverse selection. A party that does not bear the cost of a failure has little reason to prevent it.

His clearest case is a fraud study of cash machines. Where a disputed transaction put the burden of proof on the bank, as in the United States, the banks had reason to protect their systems properly; where it lay on the customer, as in Britain, Norway and the Netherlands, they became careless and epidemics of fraud followed. The control was the same technology in both places; what differed was who paid when it failed, which is the unit's thesis stated as a change of liability rather than of design.

Herley (2009) carried that reasoning to the individual: advice on passwords, phishing-site identification and certificates shields users from direct costs while loading them with indirect ones in effort, since a minute a day spent reading web addresses would cost two orders of magnitude more than all phishing losses.

The founding case is a control its users could not use

Twelve experienced email users who knew no public-key cryptography had 90 minutes to sign and encrypt a message with PGP 5.0. About one third succeeded, and about a quarter sent the secret unencrypted while believing it protected. The authors blame the interface's model.

Adams and Sasse (1999) reached a compatible conclusion from qualitative evidence. A questionnaire of 139 respondents and 30 interviews in two organisations found that half wrote a password down. Policies demanding unrelated secrets and frequent changes conflicted with their work.

The indicators users are told to check go unnoticed

Schechter, Dhamija, Ozment and Fischer (2007) tested whether the indicators people are told to check are noticed. Sixty-seven bank customers carried out banking tasks while alarming clues were introduced: the HTTPS indicators were removed, then the site-authentication image. All 63 whose responses could be verified entered their passwords after the HTTPS indicators were removed, and 23 of the 25 using their own accounts (92 per cent) entered them when the authentication image went.

The same users heed some warnings and dismiss others

Sunshine, Egelman, Almuhimedi, Atri and Cranor (2009) surveyed 409 internet users and ran a laboratory study of five warnings, three then in use and two they designed. The redesigns performed better, yet every strategy left too many participants vulnerable.

Akhawe and Felt (2013) supplied the scale that makes variation a fact. From Firefox and Chrome telemetry they observed 25,405,944 warning impressions, with users continuing through 9 to 23 per cent of the malware and phishing warnings, 33.0 per cent of Firefox's secure-connection warnings and 70.2 per cent of Chrome's.

A permission prompt poses a choice the user cannot act on, and the burden accumulates in decisions

Felt, Ha, Egelman, Haney, Chin and Wagner (2012) moved the question to permissions, where the alternative to a prompt is a default. A survey of 308 Android users and a laboratory study of 25 found 17 per cent paid attention to permissions at installation and only 3 per cent answered all three comprehension questions correctly. The choice offered was to install the application or abandon it, and answering it required reading a description most of the sample did not fully understand.

Stanton, Theofanos, Prettyman and Furman (2016) name the mechanism by which a series of reasonable decisions becomes an unreasonable load: interviewing 40 users, they found more than half alluded to security fatigue, expressed as resignation, loss of control, fatalism, risk minimisation and decision avoidance.

Contested question: is the user or the control the tractable fault?

The literature agrees that controls are often badly made. It does not agree about what follows, and three positions are legible in it.

The first holds that the control is the fault and the remedy is to stop asking. The tool nobody could use (Whitten and Tygar, 1999), the rational workaround (Adams and Sasse, 1999) and the ignored indicator (Schechter et al., 2007; Dhamija et al., 2006) all point at the control. So do the unactionable permission (Felt et al., 2012) and the warning that invites dismissal (Akhawe and Felt, 2013).

The second holds that the user is tractable and the fault is missing instruction. Kumaraguru, Sheng, Acquisti, Cranor and Hong (2010) evaluated embedded phishing training, which delivers the lesson when the user clicks a simulated link. The trained group's false negative rate fell from 0.40 to 0.11, the knowledge survived a week and longer in a field deployment, and responses to legitimate mail did not rise. Ho and colleagues (2025) test that conclusion at scale and find far less. In an eight-month randomised controlled experiment with more than 19,500 employees and ten simulated phishing campaigns, recent awareness training bore no significant relation to failure rate, embedded training made little difference, and for some content more training predicted more failures. The one training success this unit carries is the older, smaller study.

The third holds that the dispute is not yet researchable as posed. Herley (2016) shows that a claim of the form "if you do not do X you are not secure" cannot be refuted by any observation, since the counterfactual attack is unobservable. The warning studies measure clickthrough, not harm, so neither camp can show that its intervention reduced losses. Franz, Zimmermann, Albrecht, Hartwig, Reuter, Benlian and Vogt (2021) built a taxonomy of user-facing phishing interventions from a systematic literature analysis, reporting that experimental setups often lack realism, that recurring interventions have scarcely been studied, and that user effort has scarcely been considered.

What would settle it is a field study with three properties: it follows the people who click through to a measured outcome, crosses control design with warning design on one population, and reports false positives as carefully as true ones. Herley's (2016) argument applies here too, since the claim that the control is the tractable fault is refutable only against an outcome.

The three positions rest on different kinds of evidence. The control-is-the-fault position rests on laboratory and field records of use (Whitten and Tygar, 1999; Felt et al., 2012; Akhawe and Felt, 2013); the user-is-tractable position on one embedded training evaluation (Kumaraguru et al., 2010) that a larger field experiment has since failed to reproduce (Ho et al., 2025); the not-yet-researchable position on an argument about falsifiability and on a systematic review (Herley, 2016; Franz et al., 2021). The designer removes the prompt where a default will serve and measures the burden of every decision left. The strongest objection is that this unit's remedy shares the fault it names, since no cited study reports the losses avoided by the person who never saw the prompt.

Make the safe action the default, and measure what remains

Where the safe action can be made the default, make it the default and delete the prompt. Sunshine and colleagues (2009) recommend that in their own words, and the permission study implies it (Felt et al., 2012). Where a genuine decision remains, reduce the number of decisions, make the safe option easy and keep the interface consistent (Stanton et al., 2016). A default is not neutral: it is a decision made on the user's behalf, and it deserves the evidence the prompt it replaced never had.

The warning studies measure decisions rather than harm, so a default is preferred because it removes a decision whose cost is evidenced, not because its benefit has been measured.

Where the argument is heading

Franz and colleagues (2021) close their taxonomy with the field's unfinished work: setups that lack realism, interventions that repeat, and the effort users bear are what the literature has scarcely examined. The companion movement is to stop demanding decisions the system can make. Sunshine and colleagues (2009) recommended blocking unsafe connections rather than warning about them, Stanton and colleagues (2016) recommended limiting the number of decisions, and Akhawe and Felt (2013) located the lever in the warning's type. All three rest on the measurement the area lacks: whether the person who never saw the prompt is safer than the one who dismissed it.

The sources this unit rests on

• Adams, A. and Sasse, M. A. (1999). Users are not the enemy. Communications of the ACM. doi:10.1145/322796.322806 A questionnaire of 139 respondents and 30 interviews in two organisations found half wrote a password down, because policies demanding unrelated secrets and frequent changes conflicted with their work. Non-compliance answered a badly specified control.
• Akhawe, D. and Felt, A. P. (2013). Alice in Warningland: a large-scale field study of browser security warning effectiveness. 22nd USENIX Security Symposium. https://www.usenix.org/conference/usenixsecurity13/technical-sessions/presentation/akhawe Telemetry from Firefox and Chrome gave 25,405,944 warning impressions; users continued through 9 to 23 per cent of the malware and phishing warnings, 33.0 per cent of Firefox's secure-connection warnings and 70.2 per cent of Chrome's. Type, not disposition, decides the outcome.
• Anderson, R. (2001). Why information security is hard: an economic perspective. Seventeenth Annual Computer Security Applications Conference. doi:10.1109/ACSAC.2001.991552 Anderson argues that information insecurity is at least as much a consequence of perverse incentives as of weak technology, analysing it through network externalities, moral hazard and adverse selection. It relocates the failure to the arrangement that decides who pays.
• Dhamija, R., Tygar, J. D. and Hearst, M. (2006). Why phishing works. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI). doi:10.1145/1124772.1124861 Twenty-two participants judged twenty sites: the best fooled 90 per cent of them, and 15 of the 22 passed a certificate warning without hesitation. It shows that visual deception, rather than carelessness, accounted for many of the failures.
• Felt, A. P., Ha, E., Egelman, S., Haney, A., Chin, E. and Wagner, D. (2012). Android permissions. Eighth Symposium on Usable Privacy and Security (SOUPS). doi:10.1145/2335356.2335360 A survey of 308 Android users and a laboratory study of 25 found 17 per cent paid attention to permissions at installation and only 3 per cent answered all three comprehension questions correctly. The screen asked for a decision the user could not act on.
• Franz, A., Zimmermann, V., Albrecht, G., Hartwig, K., Reuter, C., Benlian, A. and Vogt, J. (2021). SoK: still plenty of phish in the sea, a taxonomy of user-oriented phishing interventions and avenues for future research. Seventeenth Symposium on Usable Privacy and Security (SOUPS). https://www.usenix.org/conference/soups2021/presentation/franz A systematic literature analysis yielded a taxonomy of phishing interventions classified by type and timing; the authors report that experimental setups often lack realism and that recurring interventions and user effort have scarcely been studied, calling the last a major shortcoming.
• Herley, C. (2009). So long, and no thanks for the externalities: the rational rejection of security advice by users. Proceedings of the 2009 workshop on New Security Paradigms (NSPW). doi:10.1145/1719030.1719050 Herley argues that advice on passwords, phishing identification and certificates shields users from direct costs while burdening them with greater indirect ones, so rejection is rational. A minute a day reading web addresses would cost two orders of magnitude more than all phishing losses.
• Herley, C. (2016). Unfalsifiability of security claims. Proceedings of the National Academy of Sciences 113(23). doi:10.1073/pnas.1517797113 He shows that claims of the form "if you do not do X you are not secure" are unfalsifiable, since the counterfactual attack is unobservable, so errors accumulate and correction runs one way only. It is the test the unit applies to its own thesis.
• Ho, G., Mirian, A., Luo, E., Tong, K., Lee, E., Liu, L., Longhurst, C. A., Dameff, C., Savage, S. and Voelker, G. M. (2025). Understanding the Efficacy of Phishing Training in Practice. 2025 IEEE Symposium on Security and Privacy (SP). doi:10.1109/SP61157.2025.00076 Ran an eight-month randomised controlled experiment with more than 19,500 employees and ten simulated phishing campaigns, finding no significant relation between recent awareness training and failure rate and little effect from embedded training. The largest test of phishing training the unit carries.
• Kumaraguru, P., Sheng, S., Acquisti, A., Cranor, L. F. and Hong, J. (2010). Teaching Johnny not to fall for phish. ACM Transactions on Internet Technology 10(2). doi:10.1145/1754393.1754396 Embedded phishing training, which delivers the lesson when a user clicks a simulated link, cut the trained group's false negative rate from 0.40 to 0.11, retained it over a week and longer in a field deployment, and did not raise responses to legitimate mail.
• Schechter, S. E., Dhamija, R., Ozment, A. and Fischer, I. (2007). The emperor's new security indicators. 2007 IEEE Symposium on Security and Privacy. doi:10.1109/SP.2007.35 Sixty-seven bank customers carried out banking tasks as alarming clues were introduced: all 63 whose responses could be verified entered their passwords after HTTPS indicators were removed, and 23 of the 25 using their own accounts (92 per cent) entered them when the authentication image went.
• Stanton, B., Theofanos, M. F., Prettyman, S. S. and Furman, S. (2016). Security fatigue. IT Professional 18(5). doi:10.1109/MITP.2016.84 Interviews with 40 users found more than half alluding to security fatigue, expressed as resignation, loss of control, fatalism, risk minimisation and decision avoidance, produced by accumulation rather than any single demand. They recommend limiting the number of decisions.
• Sunshine, J., Egelman, S., Almuhimedi, H., Atri, N. and Cranor, L. F. (2009). Crying wolf: an empirical study of SSL warning effectiveness. 18th USENIX Security Symposium. https://www.usenix.org/legacy/events/sec09/tech/full_papers/sunshine.pdf A survey of 409 internet users and a 100-participant laboratory study compared three browser warnings with two the authors designed; the redesigns performed better, yet every strategy left too many participants vulnerable. They recommend blocking unsafe connections and staying silent in safe ones.
• Whitten, A. and Tygar, J. D. (1999). Why Johnny can't encrypt: a usability evaluation of PGP 5.0. 8th USENIX Security Symposium. https://www.usenix.org/conference/8th-usenix-security-symposium/why-johnny-cant-encrypt-usability-evaluation-pgp-50 Twelve experienced email users with no public-key cryptography knowledge had 90 minutes to sign and encrypt a message with PGP 5.0; about one third succeeded and about a quarter sent the secret unencrypted. The authors blame the interface's model, not the users' ability.