Professional codes bind the member, not the employer, so their behavioural effect is weak

A professional code is usually described as the lever a practitioner pulls when a project goes wrong. The ACM and BCS codes bind a named person inside an organisation that has not signed them. They supply reasons and a standard the member will be judged against. They do not supply power over an employer. The one controlled test of a code's effect on developer decisions returns no observable effect. A code is worth keeping, and this second of ten units argues it is not the instrument that makes a system safe.

Each code binds a person, and each states its own limits

The ACM Code of Ethics and Professional Conduct was adopted by the ACM Council on 22 June 2018 and binds every ACM member, SIG member and award recipient. Principle 1.2 requires a professional to avoid harm, to consider the potential impacts of a decision on everyone affected, and to report any signs of system risk, adding that where leaders do not act it may be necessary to blow the whistle. Principle 1.6 requires only the minimum personal information necessary, with defined retention periods, and forbids reuse of personal data for another purpose without consent. The BCS Code of Conduct, version 8, approved on 8 June 2022, sets four principles: public interest, professional competence and integrity, duty to the relevant authority, and duty to the profession.

The instrument names the individual and not the employer

The BCS code states its own limit: it governs the conduct of the individual, not the nature of the business or the ethics of a relevant authority, the employer or client with authority over the member's work. A code can oblige a member to refuse a task, report a risk or leave. It cannot oblige an employer to fund a fix, change a release or delete data. Enforcement stops at the profession's edge: the BCS routes breaches to disciplinary action that can end in expulsion from membership, the ACM publishes enforcement procedures of its own, and neither sanction reaches a firm that never joined. Nor does either reach the freelance developer, the sole trader or the platform worker who belongs to no body at all, so membership is the instrument's first boundary.

A case where the division cost lives

The best documented case is still the Therac-25. Leveson and Turner (1993) investigated six known accidents between June 1985 and January 1987, in each of which the accelerator delivered massive radiation overdoses, with deaths and serious injuries; they report the description of them as the worst series of radiation accidents in the 35-year history of medical accelerators. Two findings matter here. The machine relied on software for safety functions that earlier devices handled with hardware interlocks, so the hazard was a software hazard. And a reported hazard did not produce an investigation: the authors state that the first phone call and the first lawsuit should each have triggered an immediate response. An organisation that did not act absorbed the reported risk.

A code supplies reasons, and reasons have to be taught

If a code is a source of reasons rather than of power, the question is how those reasons reach a decision. Gotterbarn (2001) argues that many software problems can be traced to a narrow understanding of professional responsibility. He proposes an expanded concept of positive responsibility. Martin and Weltz (1999) supply the pedagogical half. Teaching a code as a recitation changes nothing, and they set out a curriculum that puts the social impact of computing into technical courses. A code works when its reasons are rehearsed where the decisions are taken.

The one controlled test returns no effect

The strongest evidence is a controlled study, and it is against the code's behavioural effect. McNamara, Smith and Murphy-Hill (2018) replicated a prior behavioural ethics study. Participants were 63 software engineering students and 105 professional developers responding to 11 ethical vignettes, and some were instructed to consider the ACM code. The instruction had no observed effect against a control group. The authors ask what techniques could improve ethical decision making if a code does not. The result does not show the code is worthless. It shows that handing a member the code is not what changes the decision.

The cost falls on the member who reports

Mesmer-Magnus and Viswesvaran (2005) meta-analysed 193 correlations from 26 samples with 18,781 participants. They found that personal, contextual and wrongdoing characteristics relate more strongly to whistleblowing intentions than to actual whistleblowing, with retaliation best predicted by contextual variables. Measured intentions overstate what people do. Bjorkelo (2013) documents the price. He reports a link between whistleblowing and workplace bullying, with health effects that include depression and symptoms analogous to post-traumatic stress.

Whittaker (2021) records what happened to people who raised concerns inside the large firms that employ them: Facebook revoked data access for researchers at New York University studying the company, Google told its own researchers to strike a positive tone in findings, Amazon retaliated against critics of its products, and Google dismissed Timnit Gebru after she declined to withdraw from a paper critical of its systems. The piece argues from documented cases rather than measuring how common the cost is. It shows the mechanism a personal code cannot reach: the sanction is applied by the employer who disagrees with the reason the code supplied.

The constraint the evidence points to is organisational

Madaio, Stark, Wortman Vaughan and Wallach (2020) co-designed fairness checklists with 48 practitioners and found that checklists can supply organisational infrastructure for formalising ad-hoc processes and empowering individual advocates, while culture shapes whether they work. Vakkuri, Kemell, Jantunen and Abrahamsson (2020) studied startup-like environments where much AI software is built and found a complete ignorance of ethical consideration, naming documentation and error handling as practices that could carry ethics where they already exist. Passi and Sengers (2020) describe systems as made to work through discretionary negotiation by many actors, after six months of fieldwork with a corporate data science team. The code supplies the reason. The organisation supplies or withholds the conditions under which it is acted on.

Contested question: is the weakness a defect of the codes or a fact about organisations?

One position holds that the weakness is a defect of the instruments and their teaching, and so is fixable. Grosz, Grant, Vredenburgh and colleagues (2019) give the position its strongest institutional form. They report a Harvard-based curriculum pilot.

The opposing position holds that the weakness is structural. An instrument addressed to an individual cannot bind the organisation that decides, and the BCS code concedes as much in its own text. McNamara and colleagues (2018) tested the code as an individual-level intervention and found nothing. Madaio and colleagues (2020) locate efficacy in organisational culture.

What would settle it is a study that varies the organisation while holding the code constant. McNamara and colleagues (2018) tested the code alone, with no organisational backing, reporting channel or mandate. Their design therefore cannot separate a code that fails from a code given nothing to work with. Mittelstadt (2019) names what would need varying. Against medicine, AI development lacks common aims and fiduciary duties, professional norms, proven methods for translating principles into practice, and robust accountability mechanisms. A replication that added one of those conditions and re-ran the vignettes would separate the code's effect from the organisation's. The literature has not run it.

The two sides do not carry equal evidence. The structural side has one controlled experiment, the vignette replication of McNamara and colleagues (2018), and field evidence from Madaio and colleagues (2020) and Vakkuri and colleagues (2020). The fixable side has no experiment, only an argument from practice in Gotterbarn (2001) and Martin and Weltz (1999) and a curriculum pilot in Grosz and colleagues (2019). Olson, Fischer, Kunneman and Guzman (2025) surveyed 217 practitioners: a quarter did not feel empowered to intervene on an ethical problem, and the practitioners reporting the most concerns were those from marginalised groups. A null result from one laboratory design cannot separate the two. The strongest objection to this unit's thesis is that a code was never tested under the conditions it was written for. While the dispute is open, a designer should cite the code in writing, keep the record of the risk raised, and route it to someone able to act.

Where the argument is heading

The field is moving from principles to the organisational machinery that makes them operative, which is an admission of the thesis. Mittelstadt (2019) counts at least 84 public-private initiatives that have produced statements of principles for AI; the primary survey behind that count is Jobin, Ienca and Vayena (2019). The responses are organisational. Madaio and colleagues (2020) co-design checklists that formalise processes a team already runs informally. The third unit asks whether consent can carry the autonomy the codes cannot.

The sources this unit rests on

• ACM (2018). Code of Ethics and Professional Conduct. https://www.acm.org/code-of-ethics Adopted by the ACM Council on 22 June 2018, binding every ACM member, SIG member and award recipient. Principle 1.2 requires avoiding harm and reporting system risk; Principle 1.6 requires minimum data and defined retention. Both duties fall on a person.
• BCS (2022). Code of Conduct. https://www.bcs.org/membership-and-registrations/become-a-member/bcs-code-of-conduct Version 8, approved 8 June 2022, sets four principles: public interest, professional competence and integrity, duty to the relevant authority, and duty to the profession. It states that it governs the individual and not the ethics of a relevant authority, and routes breaches to disciplinary action that can end in expulsion.
• Leveson, N. and Turner, C. (1993). An investigation of the Therac-25 accidents. Computer. doi:10.1109/mc.1993.274940 The accident investigation of the Therac-25 accelerator: six overdoses between June 1985 and January 1987, with deaths and serious injuries. It found the machine could not detect its own overdoses and that a reported hazard did not trigger an investigation. The reporting duty failed at the organisation.
• Gotterbarn, D. (2001). Informatics and professional responsibility. Science and Engineering Ethics. doi:10.1007/s11948-001-0043-5 Argues that many software problems follow from a narrow understanding of professional responsibility, and proposes an expanded concept of positive responsibility. It treats the remedy as educational rather than legal, so a code's value lies in the reasons it supplies.
• Martin, C. D. and Weltz, E. Y. (1999). From awareness to action. ACM SIGCAS Computers and Society. doi:10.1145/382018.382028 Argues that teaching a code of ethics as a recitation changes nothing, proposing instead that the social impact of computing be integrated into technical courses. It is the pedagogical half of the claim that reasons must be practised.
• Mesmer-Magnus, J. R. and Viswesvaran, C. (2005). Whistleblowing in Organisations. Journal of Business Ethics. doi:10.1007/s10551-005-0849-1 A meta-analysis of 193 correlations from 26 samples with 18,781 participants. Personal, contextual and wrongdoing characteristics relate more strongly to whistleblowing intentions than to actual behaviour. Intention overstates action.
• Bjorkelo, B. (2013). Workplace bullying after whistleblowing. Journal of Managerial Psychology. doi:10.1108/02683941311321178 Reviews empirical research linking whistleblowing to subsequent workplace bullying, with health effects including depression and symptoms analogous to post-traumatic stress. It prices the personal cost of the reporting duty.
• McNamara, A., Smith, J. and Murphy-Hill, E. (2018). Does ACM's code of ethics change ethical decision making in software development? Proceedings of ESEC/FSE 2018. doi:10.1145/3236024.3264833 Replicates a behavioural ethics study with 63 software engineering students and 105 professional developers responding to 11 ethical vignettes, some instructed to consider the ACM code. The instruction had no observed effect against a control. It is the unit's one controlled test.
• Jobin, A., Ienca, M. and Vayena, E. (2019). The global landscape of AI ethics guidelines. Nature Machine Intelligence. doi:10.1038/s42256-019-0088-2 The primary survey behind the count of 84 guideline documents the unit reports. It finds a global convergence on five principles, transparency, justice and fairness, non-maleficence, responsibility and privacy, with substantive divergence about how each is interpreted.
• Mittelstadt, B. (2019). Principles alone cannot guarantee ethical AI. Nature Machine Intelligence. doi:10.1038/s42256-019-0114-4 Argues that convergence on principles resembling medical ethics is not enough, because AI development lacks common aims and fiduciary duties, professional norms, proven methods for translating principles into practice, and robust accountability mechanisms. It counts at least 84 public-private initiatives that produced principle statements.
• Olson, L., Fischer, R. A.-L., Kunneman, F. and Guzman, E. (2025). Who Speaks for Ethics? How Demographics Shape Ethical Advocacy in Software Development. arXiv preprint 2504.10276. https://arxiv.org/abs/2504.10276 A survey of 217 practitioners. A quarter did not feel empowered to intervene on an ethical problem, and marginalised practitioners reported the most concerns and the most capacity to act, which locates the constraint in organisational support. A preprint, so the findings are not yet peer reviewed.
• Madaio, M. A., Stark, L., Wortman Vaughan, J. and Wallach, H. (2020). Co-Designing Checklists to Understand Organisational Challenges and Opportunities around Fairness in AI. Proceedings of CHI 2020. doi:10.1145/3313831.3376445 A co-design process with 48 practitioners building an AI fairness checklist. It found that checklists can supply organisational infrastructure for formalising ad-hoc processes and empowering individual advocates, while culture shapes their efficacy, so the constraint sits in the organisation.
• Vakkuri, V., Kemell, K. K., Jantunen, M. and Abrahamsson, P. (2020). This is Just a Prototype. Lecture Notes in Business Information Processing. doi:10.1007/978-3-030-49392-9_13 A multiple case study of ethics in startup-like environments where much AI software is developed, finding a complete ignorance of ethical consideration. It names documentation and error handling as existing practices that could carry ethics.
• Passi, S. and Sengers, P. (2020). Making data science systems work. Big Data and Society. doi:10.1177/2053951720939605 Six months of ethnographic fieldwork with a corporate data science team. It describes systems as made to work through discretionary negotiation by many actors, so values are settled in ordinary team decisions.
• Grosz, B. J., Grant, D. G., Vredenburgh, K. and colleagues (2019). Embedded EthiCS. Communications of the ACM. doi:10.1145/3330794 Reports a Harvard-based pilot that integrates sessions on ethical reasoning throughout the computer science curriculum. It is the strongest institutional form of the position that a code's weak effect is a defect of teaching.
• Whittaker, M. (2021). The steep cost of capture. Interactions. doi:10.1145/3488666 Argues that dependence on corporate resources gives a few firms leverage over the direction of research, and documents retaliation against those who publish findings the firms dislike, including the dismissal of Timnit Gebru and the revocation of data access for outside researchers. It prices the cost of raising a concern inside the employer.