Ethics frameworks decide what not to build, and they disagree about why

Ethics frameworks are usually judged by the verdicts they reach. This first of ten units argues that test is wrong. Put a design decision to the three dominant traditions and they often return the same answer: do not ship the feature. Consequentialism points to the harm over a year of use. Deontology points to a right that is violated whatever the consequences. Virtue ethics points to what a practitioner of good character would put their name to. They agree about what a designer should not ship and disagree about why, and that disagreement is the useful part. It exposes which justification a decision rests on, and whom the team must answer.

Computer ethics begins where the older frameworks ran out of words

Moor (1985) gives the reason the disagreement is structural rather than accidental. He defines computer ethics as the analysis of the policy vacuums and conceptual muddles that new computing creates. On his account the subject is not an existing ethics applied to a new object but the response to questions no earlier framework anticipated. A policy vacuum is a case where no rule yet exists; a conceptual muddle is one where the words to state a rule are missing. Both arrive before the law. A framework must therefore be chosen and defended rather than looked up.

In January 2012 Facebook altered the emotional content of the News Feed for 689,003 members for a week and published the result as evidence that moods spread through a network (Kramer, Guillory and Hancock 2014). The authors recorded the work as consistent with a data use policy every member had accepted, and a university review board decided the project lay outside its human-subjects programme (Verma 2014). Nothing was broken, because the platform ran the experiment on its own members.

The three traditions converge on verdicts and diverge on reasons

1. Consequentialism judges an act by its outcomes. Its question is what the act brings about. Its burden is to say whose outcomes count, over what horizon, and in which setting, since Birhane (2020) shows Western-built systems imported into African contexts meeting little local scrutiny of the values they carry.
2. Deontology judges an act by duties and rights that hold whatever follows. Its question is what is owed to whom. Its burden is to resolve duties that cannot all be met.
3. Virtue ethics asks what a person of good character would do. Its question is the disposition behind the act. Its burden is to say whose character, formed where, is the measure.

Bynum (2006) carries the virtue arm into contemporary technology, setting out a virtue-shaped theory with Aristotelian roots, split into a human-centred part and one reaching every physical entity. Vallor (2016) argues for the cultivation of technomoral virtues and closes with twelve of them applied to social media, surveillance, robotics and biomedical enhancement.

Take a decision all three refuse: a notification sent at two in the morning because late-night sending lifts a re-engagement metric. The verdicts coincide again, and what differs is the challenger the reason answers: a regulator for the consequentialist, the person woken for the deontologist, the professional body for the virtue framing.

The design turn makes ethics a property of the artefact

Friedman (1996) moved the argument from verdict to specification. Value sensitive design treats a value such as autonomy or informed consent as something a design can be specified to support. It pursues that support through conceptual, empirical and technical investigations. Ethics becomes a property of the product rather than a verdict on it, and this is the position the rest of the unit answers.

Friedman, Howe and Felten (2002) applied the method to a shipped case, cookie management in the Mozilla browser. They derived four design goals from criteria for informed consent online. They moved between those goals, new technical mechanisms and formative evaluation. The mechanisms that survived were peripheral awareness of cookies and just-in-time interventions. The value reached a build rather than a memorandum.

Critics fault a missing theory, and its supporters narrow the claim

The method's critics concentrate on what it does not decide. Le Dantec, Poole and Wyche (2009) argue that its mechanics create tensions with value sensitivity, naming value classifications, weak guidance on empirical tools, and the ordering of the design process: value discovery, on their account, is an empirical and contested activity, not a reading from a list. Manders-Huits (2010) finds five shortcomings in the method, among them the risk of the naturalistic fallacy, an undetermined concept of values, and no explicit theory for value trade-offs.

Borning and Muller (2012) argue the method overclaims. They propose four next steps, among them tempered claims about universal values and contextualised value lists.

The commitment survives the work, and the method outputs a specification

Shilton (2012) supplies the evidence that a commitment survives contact with ordinary work. Her ethnography of a laboratory building mobile software to track users' locations and habits finds that practices she calls values levers let engineers agree on social values as important to design, because they open the conversation during the work. Brey (2012) supplies the stage: anticipatory technology ethics studies issues at the research and introduction stage, at the level of the technology, the artefact and the application. van Wynsberghe (2012) shows the method producing a specification rather than a principle, building care centred value sensitive design so that care robots can be evaluated retrospectively and prospectively.

A fourth framework the typology cannot hold

The three-tradition picture is tidy, and the literature does not stay tidy. Floridi (2013) treats the informational environment, not only human agents, as the bearer of moral value. A framework of that shape does not map onto the three offices, so the choice of framework is not a choice among three. Zhu, Yu, Halfaker and Terveen (2018) show where the design turn now points. Value sensitive algorithm design brings stakeholders' tacit knowledge and explicit feedback into the early stages of algorithm creation.

Contested question: does running the frameworks decide anything?

The literature disagrees about whether a pluralist method reaches a decision or only relocates it. One position holds that a method can carry values into a specification. Friedman (1996) is its origin and van Wynsberghe (2012) its strongest worked case. Her framework outputs a set of design requirements for care robots rather than a restatement of care ethics, so running the frameworks produces something to build to.

The opposing position holds that the method names values without deciding between them. Manders-Huits (2010) makes the decisive objection in her fifth shortcoming: value sensitive design has no explicit ethical theory for value trade-offs, and the concept of values is left undetermined. Le Dantec and colleagues (2009) show the same gap at the level of process. Borning and Muller (2012) confirm it from inside the tradition. When two values conflict, on this view, the method supplies the vocabulary for the conflict and not the rule that resolves it. Running several frameworks then multiplies the vocabularies without adding the rule.

What would settle it is a comparison. The question is whether a multi-framework method makes decisions more defensible, or produces better outcomes, than one framework alone. Manders-Huits's (2010) call for an explicit ethical starting point is the hypothesis the test would need. A study carrying one product through both routes would answer the question. Gerdes and Frandsen (2023) supply part of it: their review of almost three decades of value sensitive design included 54 studies that conduct or discuss technical investigations, and found little engagement at the technical level of the community, which they call a problem for the approach.

The two sides do not carry equal evidence. The affirmative side is a method with two applied cases, Friedman (1996) in the Mozilla browser and van Wynsberghe (2012) in care robotics. The sceptical side is an argument about a missing theory, from Manders-Huits (2010) and Le Dantec and colleagues (2009), and conceded in part by Borning and Muller (2012). Two applied cases against a conceptual gap and a review of 54 studies is not a comparison. The strongest objection to this unit's thesis is that the reason a team files is chosen after the decision, so the framework names a justification rather than causing what was built. While the dispute is open, a designer should record which framework supplied the reason and which audience that reason answers.

Where the argument is heading

The recent work moves the design turn towards a fixed set of norms, which sharpens rather than resolves the contested question. Umbrello and van de Poel (2021) argue that machine learning poses challenges specific to value sensitive design and propose a modified approach that integrates AI for social good principles as design norms and attends to transparency, explicability and accountability. The argument heads towards the convergence the thesis predicts and cannot yet demonstrate: shared norms, held for different reasons, whose reasons the next generation of toolkits must keep visible. The second unit asks what a code can do where the reasons cannot.

The sources this unit rests on

• Moor, J. H. (1985). What is computer ethics? Metaphilosophy. doi:10.1111/j.1467-9973.1985.tb00173.x Defines computer ethics as the analysis of the policy vacuums and conceptual muddles new computing creates, so the field answers questions no earlier framework anticipated. It grounds the claim that a framework must be chosen and defended, not looked up.
• Friedman, B. (1996). Value-sensitive design. Interactions. doi:10.1145/242485.242493 Introduced value sensitive design, treating autonomy or informed consent as something a design can be specified to support through conceptual, empirical and technical investigations. It marks the shift from verdict to property.
• Bynum, T. W. (2006). Flourishing Ethics. Ethics and Information Technology. doi:10.1007/s10676-006-9107-1 Sets out a virtue-shaped theory with Aristotelian roots, split into a human-centred part and a general part extending to every physical entity. It supplies the flourishing account the virtue arm rests on.
• Le Dantec, C. A., Poole, E. S. and Wyche, S. P. (2009). Values as lived experience. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. doi:10.1145/1518701.1518875 Argues the mechanics of value sensitive design create tensions with value sensitivity, naming value classifications, weak guidance on empirical tools, and the ordering of the design process. It shows value discovery is contested, not a reading from a list.
• Manders-Huits, N. (2010). What Values in Design? The Challenge of Incorporating Moral Values into Design. Science and Engineering Ethics. doi:10.1007/s11948-010-9198-2 Finds five shortcomings in value sensitive design, including the naturalistic fallacy risk, an undetermined concept of values, and no theory for value trade-offs. It is the unit's sharpest statement that the method names values without deciding between them.
• Borning, A. and Muller, M. (2012). Next steps for value sensitive design. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. doi:10.1145/2207676.2208560 Argues value sensitive design overclaims and proposes four next steps: tempered claims about universal values, contextualised value lists, stronger participants' voices, and a clearer researchers' voice. It narrows the method's claims from inside.
• Shilton, K. (2012). Values Levers. Science, Technology, and Human Values. doi:10.1177/0162243912436985 An ethnography of a laboratory engineering mobile phone software to track users' locations, habits and behaviours. It finds that practices it names values levers let engineers agree on social values as important to design.
• Brey, P. A. E. (2012). Anticipating ethical issues in emerging IT. Ethics and Information Technology. doi:10.1007/s10676-012-9293-y Presents anticipatory technology ethics, which studies issues at the research and introduction stage by anticipating future devices, applications and consequences, at the levels of technology, artefact and application. It gives the frameworks a stage before any rule exists.
• van Wynsberghe, A. (2012). Designing Robots for Care. Science and Engineering Ethics. doi:10.1007/s11948-011-9343-6 Builds care centred value sensitive design from the value sensitive design blueprint, using care values as the foundational values and care-ethics elements as the normative criteria. It is the clearest worked case, producing a specification rather than a principle.
• Floridi, L. (2013). The Ethics of Information. Oxford University Press. doi:10.1093/acprof:oso/9780199641321.001.0001 The first philosophical monograph dedicated to information ethics, laying its foundations with a metatheoretical, an introductory and an analytic aim. It makes the informational environment, not only human agents, the bearer of moral value, so its shape does not map onto the three traditions.
• Vallor, S. (2016). Technology and the Virtues. Oxford University Press. doi:10.1093/acprof:oso/9780190498511.001.0001 Argues for cultivating technomoral virtues as qualities of character needed to live well in a changing technosocial environment, drawing on Aristotle, Confucianism and Buddhism, and closes with twelve such virtues. It carries the virtue arm into contemporary technology.
• Zhu, H., Yu, B., Halfaker, A. and Terveen, L. (2018). Value-Sensitive Algorithm Design. Proceedings of the ACM on Human-Computer Interaction. doi:10.1145/3274463 Proposes value sensitive algorithm design, which brings stakeholders' tacit knowledge and explicit feedback into the early stages of algorithm creation so that choices can avoid bias. It applies the design turn to an algorithm rather than an artefact.
• Umbrello, S. and van de Poel, I. (2021). Mapping value sensitive design onto AI for social good principles. AI and Ethics. doi:10.1007/s43681-021-00038-3 Argues machine learning poses challenges specific to value sensitive design and proposes a modified approach that integrates AI for social good principles as design norms and attends to transparency, explicability and accountability.
• Gerdes, A. and Frandsen, T. F. (2023). A systematic review of almost three decades of value sensitive design (VSD): what happened to the technical investigations? Ethics and Information Technology. doi:10.1007/s10676-023-09700-2 Screened 2,779 publications and included 54 studies that conduct or discuss technical investigations in value sensitive design from 1996 to 2023. It found little engagement at the technical level of the community, which turns the unit's sceptical position into a measured finding.
• Kramer, A. D. I., Guillory, J. E. and Hancock, J. T. (2014). Experimental evidence of massive-scale emotional contagion through social networks. PNAS. doi:10.1073/pnas.1320040111 Altered the emotional content of the News Feed for 689,003 members for one week in January 2012 and reported evidence that moods spread through a network, with effect sizes as small as d = 0.001. It is the unit's instance of a policy vacuum, and the expression of concern is Verma (2014), https://pmc.ncbi.nlm.nih.gov/articles/PMC4115552/
• Friedman, B., Howe, D. C. and Felten, E. (2002). Informed consent in the Mozilla browser. Proceedings of HICSS-35. doi:10.1109/HICSS.2002.994366 Applied value sensitive design to a shipped system, deriving four design goals from criteria for informed consent online and building peripheral awareness of cookies and just-in-time interventions, so the design turn reaches a build rather than a memorandum.