Ethics, privacy, law and responsible design
HCI3106 · 15 credits · Level 6
The obligations that attach to designing systems people must live with, from consent and data protection to dark patterns and the AI Act.
Ten units
- 1Ethics frameworks decide what not to build, and they disagree about whyThe three normative traditions converge on what a designer should not ship while diverging on the reason, so running all three has practical value only because it exposes which justification a decision already rests on, and to whom that justification has to be defensible.
- 2Professional codes bind the member, not the employer, so their behavioural effect is weakThe ACM and BCS codes constrain an individual inside an organisation that need not share them, which makes them a source of reasons and a standard to be judged against rather than leverage; the one controlled test of a code's effect on developer decisions returns no effect, which locates the binding constraint in the organisation.
- 3Consent fails as an autonomy mechanism because the choice and the preference are both unstableThe privacy paradox is better read as an artefact of measurement and of context-dependent norms than as a fact about careless users, so a lawful consent screen can record agreement while failing autonomy, and the repair belongs in the choice architecture and the flow of information rather than in clearer wording.
- 4The law binds through named duties, and what a right to explanation obliges is still contestedRead operationally the UK regime is a short list of duties a design team can discharge, with the impact assessment as its one real design instrument, while the unresolved fight is whether the data protection provisions require an explanation of an automated decision at all; and the AI Act makes classification rather than principle the point at which a case is decided.
- 5Dark patterns are measurable, and the taxonomy and the law disagree about which members are unlawfulCrawls and controlled experiments established that deceptive design is common and effective and that disclosure does not neutralise it, which moved the question from ethics to enforcement, but the taxonomy that made it measurable is a designer's list and the regulators now have to decide which entries are unlawful rather than merely unpleasant.
- 6Manipulation is defined by the mechanism of influence, not by the outcome or the user's reported feelingPersuasion offers reasons a person can accept, manipulation bypasses or subverts those reasons and coercion removes the choice, so the test is a judgement about mechanism rather than a measurement; the persuasive technology tradition assumed the change was wanted, and the personalisation that followed is what made that assumption untenable.
- 7Surveillance is an asymmetry of seeing before it is a technology, and monitoring is where a graduate meets itThe ethical problem is the unequal relation between the party who observes and the party observed rather than the sensor, which is why an unnamed lawful basis, a default retention period and an undisclosed reuse are the failures that matter; the outcome evidence once lagged the legal argument, and the 2023 meta-analysis now reports stress with no performance gain.
- 8Fairness has no single definition, so an audit reports a value choice rather than a verdictThe impossibility results and the deployed cases together show that fairness criteria conflict, that the label often carries the bias rather than the model, and that choosing a metric is choosing which error will fall on whom; an audit can establish that performance differs across groups but not that a system is fair, and the auditor's access is the binding limit.
- 9Risk tiering turns the ethical question into a classification question, and accountability needs an institution rather than a documentThe AI Act attaches its obligations to the risk tier rather than to the principle, so the contested work moves into the definition of the system, and the proposals for accountability that survive the evidence are the ones that place a function inside the organisation rather than producing another document.
- 10Accessibility is an anticipatory duty, and conformance is not the dutyThe equality duty requires reasonable steps taken in advance, while conformance standards answer a different question, so a product can pass an automated audit and still fail the person who needs it; the social-model literature explains why the duty attaches to the design rather than to the user, and participatory method is what that literature asks for and rarely measures.
10 of 10 units written. Rows marked in preparation have no body yet.
Specification
| Assessment | Weight | Type |
|---|---|---|
| Data protection impact assessment | 35% | coursework |
| Dark pattern audit and regulatory analysis | 35% | coursework |
| Ethical case analysis and position paper | 30% | coursework |
What you will be able to do
- Apply a data protection impact assessment to a product and act on its findings.
- Distinguish persuasion from manipulation in a concrete interface and justify the judgement.
- Identify dark patterns in a deployed product and state the legal exposure.
- Classify a system under the EU AI Act risk tiers and name the resulting obligations.
- State a professional position on a contested design and defend it against the commercial case.
The reading this module rests on
- ACM (2018). Code of Ethics and Professional Conduct. Short, specific, and usefully uncomfortable.
- Gray, C. et al. (2018). The dark (patterns) side of UX design. CHI 2018. The taxonomy, written by researchers who found it in practice.
- Mathur, A. et al. (2019). Dark patterns at scale: findings from a crawl of 11K shopping websites. CSCW 2019. The measurement study that made the problem undeniable.
- Acquisti, A., Brandimarte, L. and Loewenstein, G. (2015). Privacy and human behavior in the age of information. Science 347(6221), 509-514. Why the privacy paradox is less paradoxical than it sounds.
- Regulation (EU) 2024/1689 (Artificial Intelligence Act), with a practitioner's guide to the risk tiers. Read the definitions, not the whole instrument.
Ethics, privacy, law and responsible design
Most interaction design asks how to build something well. Ethics asks a harder and earlier question: whether it should exist in this form at all. That question is answered by small decisions that are expensive to reverse later. A default setting, a retention period, a notification sent at two in the morning. Each one looks technical, and each one distributes benefit and burden.
What follows is not a list of virtues. It is the machinery a practitioner uses to decide what not to build, and to defend that decision afterwards. The law matters because it turns some obligations into enforceable duties. It does not cover everything that matters, and the gap between the law and what is right is where professional judgement lives.
Ethics decides what not to build, and the framings disagree about why
Three traditions do most of the work, and they can be applied rather than recited. Consequentialism judges an act by its outcomes. Deontology judges it by duties and rights that hold whatever the outcome. Virtue ethics asks what a person of good character would do. The three often agree on the verdict and disagree on the reason, which matters when you must justify a decision to someone who does not share your reasoning.
The contested question here is whether ethics adds anything to law. One position holds that a compliant product is an ethical one, because the law already encodes society's judgements. The other holds that law lags practice by years, so the hard cases arrive before any rule exists. The second position has the stronger practical support: a system can be lawful and still cause harm that no statute anticipated.
Consider a feed ranked for engagement and offered to teenagers. Every step in it may be lawful. The consequentialist asks what the ranking does over a year of use. The deontological framing asks whether the design respects the user's capacity to choose. The virtue framing asks what a practitioner who cared about those users would ship. Running all three exposes which reason you are really relying on.
Codes of conduct are instruments, and instruments have consequences
The ACM Code of Ethics and Professional Conduct was adopted by the ACM Council on 22 June 2018. Read it as an instrument rather than a slogan. Principle 1.2 requires a computing professional to avoid harm and to report signs of system risk, and it names whistleblowing as a last resort when leaders will not act. Principle 1.6 requires the minimum personal information necessary, defined retention periods, and no reuse of data for a new purpose without consent. Section 4 sets out enforcement.
The BCS Code of Conduct states four principles for members: public interest, professional competence and integrity, duty to the relevant authority, and duty to the profession. Its duty to the relevant authority forbids misrepresenting or withholding information about the performance of products or systems.
The two codes differ in reach. The ACM Code binds members and award recipients and has enforcement procedures. The BCS code binds members and supports complaints about misconduct. Neither binds an employer, so a code constrains an individual inside an organisation that may not share it.
Take a team told to describe a release as improving privacy when it has widened data sharing. The BCS duty and ACM Principle 1.3 both bite on the wording, not on the product. The honest limit is that codes are weakly enforced, and there is little evidence that they change behaviour on their own.
A consent screen is not agreement
Autonomy means a person decides for themselves, with the information and the freedom to do so. Consent is the mechanism that carries autonomy into data processing. A screen that records a click is not the same thing as agreement, and the law now separates the two. Under the UK GDPR, consent must be freely given, specific and informed, and signalled by an unambiguous indication.
Acquisti, Brandimarte and Loewenstein review the behavioural evidence under three themes. People are uncertain about the consequences of disclosure and about their own preferences. Concern about privacy depends on context. And that concern is malleable, capable of being manipulated by commercial and governmental interests. The third theme is why the so-called privacy paradox is less paradoxical than the phrase suggests.
A cookie banner with a prominent accept button and a buried refusal path is the worked case. The design collects consent without offering a real choice, since refusal is technically present and practically costly. The weak part of the evidence is that much of it comes from behavioural studies with modest effect sizes. The malleability claim also functions as a research agenda as much as a settled finding.
The privacy that binds a UK practitioner is written as duties
The law a UK practitioner works under is the UK GDPR read with the Data Protection Act 2018. Its operational content is a short list. You need a lawful basis for processing, and consent is only one of seven. Purpose limitation stops you reusing data for a purpose you never gave. Data minimisation requires you to collect what the purpose needs and no more. Storage limitation requires a retention period you can defend. The right of access lets a person ask what you hold, and get it.
The data protection impact assessment is the design instrument in this regime. You complete one before processing that is likely to result in high risk, and it is a process rather than a form. Describe the processing. Consult where the risk is serious. Assess necessity and proportionality. Identify the risks to individuals, record the mitigations, and then decide whether to proceed.
Worked case: a university wants a dashboard that predicts which students are at risk of failing. The assessment forces the questions the feature brief avoided, including whether the prediction changes how a tutor treats a named student. The ICO states that this guidance is under review following the Data (Use and Access) Act, so treat any summary of it as provisional.
Persuasion, manipulation and coercion are three different claims
Fogg's persuasive computers paper of 1998 founded a research programme on machines that change what people think and do, and he later named the area captology. Read the tradition on its own terms. It assumed the change was wanted, as in health or sustainability work.
The three terms are not degrees of one thing. Persuasion changes behaviour through reasons the person can accept. Manipulation changes behaviour by bypassing or subverting the person's reasoning. Coercion removes the choice altogether. The distinction is a judgement about the mechanism, not a measurement you can take from a screen.
Worked case: a free trial that converts to a paid subscription unless the user telephones a support line. Nothing is hidden, and the terms are lawful. The cancellation route imposes a deliberate cost, which places it on the manipulation side of the line.
Gray and colleagues found that practitioners used the umbrella term dark patterns for a wide range of ethical concerns. They also reported a shared worry that designers could easily become complicit in manipulative or unreasonably persuasive work. That worry is evidence that the line exists and is difficult to hold.
Dark patterns are measurable, and measurement has limits
Gray and colleagues assembled practitioner-identified examples and analysed them by content analysis. The result separates cases that practitioners had conflated. Mathur and colleagues then measured the problem at scale. Their automated crawl of about 53,000 product pages from about 11,000 shopping websites found 1,818 dark pattern instances, covering 15 types in 7 broader categories. They classified 183 websites as using deceptive practices, and found 22 third-party entities selling dark patterns as a turnkey service.
Worked case: a checkout with a countdown timer and a claim that only two items remain. Both are scarcity and urgency patterns. A crawl can find them and count them, which is why the finding moved the argument from anecdote to prevalence. Both teams called for regulators to act, and regulation has begun to arrive.
The limits matter. A crawl reads the page it was served and cannot observe patterns that are personalised or tested in an experiment, so an audit measures one version of a product. Raji and colleagues make a related point about artificial intelligence. Auditing a deployed system from outside the organisation is difficult, so their framework moves the work inside, across the whole development life cycle, with documents that together form an audit report.
Surveillance is an asymmetry before it is a technology
Surveillance is a relation of unequal seeing. One party observes, the other is observed, and the imbalance grows with every extra data source. The asymmetry, not the camera, is the ethical problem.
Workplace monitoring is the case a graduate is most likely to meet. Keystroke logging, screen capture, location tracking and productivity scoring all process personal data. The employer is therefore bound by the UK GDPR, and the ICO publishes guidance on workers' information. The ACM Code requires the minimum personal information necessary and defined retention periods, which is precisely what productivity scoring tends to violate.
A concrete case is software that screenshots a remote worker's screen every ten minutes and scores their activity. The data is often collected without a lawful basis the employer can name. It is then kept by default and reused for performance decisions nobody disclosed. Principle 1.2 also warns about harm that arises from data aggregation and emergent system properties, where combined data reveals more than any single field.
The honest position is that this evidence base is thin. Surveillance research in HCI is fragmented, and much of it studies attitudes rather than outcomes.
The AI Act sorts systems by risk, and the sorting is the obligation
Regulation (EU) 2024/1689, the Artificial Intelligence Act, was adopted in 2024 and is in force. It applies from 2 August 2026. Its prohibitions and general provisions apply from 2 February 2025, and its obligations for general-purpose models from 2 August 2025. Article 5 prohibits unacceptable practices. These include systems that deploy subliminal, manipulative or deceptive techniques, and systems that exploit the vulnerabilities of a person or a group. They include social scoring and real-time biometric identification in publicly accessible spaces.
High-risk systems carry the heaviest duties: risk management, data governance, technical documentation, logging, human oversight, and standards for accuracy and robustness. The recitals name education, credit scoring, and judicial and electoral assistance among the high-risk domains.
Worked case: a tool that screens curricula vitae and shortlists applicants. Classifying it as high risk triggers documentation and oversight duties. In practice the contested argument is whether a system is high risk at all, so the classification is where the compliance fight happens.
Two limits. The Act is European Union law, so a UK practitioner meets it through market access rather than through UK statute. And a documentation regime produces documents, which is not the same as a safer system.
Accessibility has legal teeth, and raising a concern has a price
Accessibility is an ethical obligation that already carries legal force. The Equality Act 2010 requires reasonable adjustments. Section 20 sets three requirements. A provision, criterion or practice can put a disabled person at a substantial disadvantage. So can a physical feature, or the absence of an auxiliary aid. In each case the responsible person must take reasonable steps to avoid the disadvantage. Where information is involved, section 20(6) says reasonable steps include providing it in an accessible format. Section 29 applies the duty to service providers and to public functions, and section 149 places a public sector equality duty on public authorities.
Worked case: a council's online benefits claim form. The duty is anticipatory, so the council cannot wait for a blind claimant to complain before acting. A conformance checklist is not the duty, because section 20 asks what is reasonable in the circumstances.
Professional responsibility closes the argument. Both codes require a practitioner to raise a concern, and ACM Principle 1.2 names whistleblowing when leaders will not act. It also warns that careless reporting can itself cause harm, so the duty is to assess before you report. The cost of doing so is real and often personal, which is why organisations that reward silence get silence.
The next step is to carry a defensible position into paid work, where clients, contracts and colleagues test it.