Year 3 / Module 22 Ethics, privacy, law and responsible design / Unit 4 / The law binds through named duties, and what a right to explanation obliges is still contested

The law binds through named duties, and what a right to explanation obliges is still contested

HCI3106 · Unit 4 of 10

Read operationally the UK regime is a short list of duties a design team can discharge, with the impact assessment as its one real design instrument, while the unresolved fight is whether the data protection provisions require an explanation of an automated decision at all; and the AI Act makes classification rather than principle the point at which a case is decided.

The law binds through named duties, and what a right to explanation obliges is still contested

The first answer the law invites is a principle: be fair, be transparent, respect the person. What a team can discharge and be held to is a duty with a name. Read that way, the regime shortens to a lawful basis, the minimum your purpose needs, and a defensible retention period. This fourth of ten units argues that the regime is operational, that the impact assessment is its design instrument, and that the dispute is not whether the law binds but what one duty obliges. The unsettled question is whether the provisions on automated decisions require an explanation of a particular decision. A second instrument writes a version of the duty: Article 86 of the Artificial Intelligence Act gives a person affected by a decision taken on a high-risk system's output the right to clear and meaningful explanations of its role, and Arnesen (2026) finds it more explicit and narrower. Classification under that Act is argued where this course reads its risk tiers.

Read as an operating manual, the regime is a short list of duties

The instrument is Regulation (EU) 2016/679, retained in the United Kingdom as the UK GDPR and read with the Data Protection Act 2018 (UK Parliament 2018). The duties sit in Article 5: lawfulness, fairness and transparency, purpose limitation, data minimisation, accuracy, and storage limitation.

The Information Commissioner's Office's guide to lawful basis (ICO 2024) states that Article 6 offers seven lawful bases and that consent is only one of them. The office updated it on 2 April 2026 to reflect the Data (Use and Access) Act 2025 (UK Parliament 2025). It now carries a basis for recognised legitimate interests.

The impact assessment is the one instrument that acts on the design

The impact assessment governs what a team builds. The ICO's guidance (ICO 2024) sets out when an assessment is required, what it must contain and when the regulator must be consulted. Read as a checklist that is bureaucratic. Read as an argument it is the only place in the regime where someone must say in advance what the system will do to a person.

Edwards and Veale (2017) argue the better leverage against algorithmic harm lies in privacy by design, impact assessment and certification.

The Dutch government ran SyRI, a system that combined benefits, tax and housing records to flag households. In February 2020 the District Court of The Hague held that the authorising legislation failed the fair balance Article 8 of the European Convention on Human Rights requires. The use of the system was not sufficiently transparent or verifiable, and the court declared the provisions to have no binding effect (Rechtbank Den Haag 2020). The failure the court named is the one an impact assessment exists to prevent. Van Bekkum and Zuiderveen Borgesius (2021) note that the immediate effects were narrow because the judgement turned on the circumstances of the case.

What the duties owe a person is information about a decision, not an explanation of it

Articles 13 to 15 give a person the right to be told that automated decision-making exists, and to receive meaningful information about the logic involved, with the significance and the envisaged consequences of the processing. The Article 29 guidelines (2018) read that duty alongside Article 22, which restricts solely automated decisions. The ICO's guidance on AI and data protection (ICO 2024) restates the Article 22 questions.

The distinction between an explanation of the system and an explanation of a decision is what the dispute turns on. A notice about the model's logic answers a question about a system, while a refused person wants to know why this decision happened to them. The United Kingdom's implementation has moved the first duty closer to the second. Article 15(1)(h), as it stands after the 2025 Act, ties that disclosure to the cases where safeguards are required under the new Article 22C. It limits subject access to what a controller can provide on a reasonable and proportionate search. A right exercised by a request assumes a device, a connection and the literacy to use both, and van Deursen and van Dijk (2019) find that device diversity and the ongoing cost of hardware and subscriptions keep shaping skills, uses and outcomes even where a connection is universal.

Contested question: does the regime require an explanation of an individual decision?

Wachter, Mittelstadt and Floridi (2017) hold that it does not, and their paper is the sceptical reference. Reading the text, they conclude that the Regulation grants not an explanation of specific decisions but what they name a right to be informed. That right is limited information under Articles 13 to 15 about the existence of automated decision-making, the logic involved, and its significance and consequences. Article 22, they add, is ambiguous and thinly safeguarded, so the regime risks being toothless.

Selbst and Powles (2017) hold the opposite. They argue that Articles 13 to 15 confer rights to meaningful information about the logic involved, which is a right to explanation whether the phrase is used or not. Casey and colleagues (2019) add that investigative, corrective and punitive powers make supervisory authorities the de facto interpreters.

A third position refuses the question. Edwards and Veale (2017) argue that even a generous reading is unlikely to remedy the harms that drive the demand. The fit between legal and technical notions of explanation is poor, and the search has produced a transparency fallacy. Malgieri (2019) shows the question has no single answer across jurisdictions. Nine member states wrote specific exemptions and safeguards, with scope ranging from decisions with legal or similarly detrimental effects to any decision with a significant impact, and most copied the three safeguards. France and Hungary added a right to legibility, and Ireland and the United Kingdom regulated human intervention.

What would settle it is a decision by an authority with power to interpret the text, or an amendment by a legislature. The second has happened, and it did not resolve the question in either camp's terms.

The two camps do not carry equal evidence, because both read the same text. Wachter, Mittelstadt and Floridi (2017) is a close reading of Articles 13 to 15 and 22, Selbst and Powles (2017) a reply on the same provisions, and Edwards and Veale (2017) an argument about remedies rather than words. Malgieri (2019) supplies the one comparative check, nine member states writing their own safeguards. None of them had outcome data, and the 2025 Act replaced the text they argued over. The strongest objection to this unit's thesis is that a duty no regulator enforces is a duty in name only, so naming the duties overstates what binds a team. While the dispute is open, a designer should write down, for each automated decision, what it did, who could contest it, and what record survives.

A statute settled the question by writing a different duty

Section 80 of the Data (Use and Access) Act 2025 substitutes Article 22 of the UK GDPR with Articles 22A to 22D.

A decision is based solely on automated processing if there is no meaningful human involvement in the taking of the decision.

A decision rubber-stamped at a screen is no longer automated, and one made without such a person is. It also moves the question from the interface, where a designer works, into the pipeline, where the decision is taken. The substituted Articles require safeguards where a significant decision is based solely on automated processing. The person is informed, may make representations, and may contest the decision. A regulation-making power governs when there is or is not meaningful human involvement. The substitution came into force on 19 June 2025 for specified purposes and in full on 5 February 2026.

Three consequences follow. First, the settlement is procedural: Parliament did not write an explanation duty, but a list of things a controller must be able to do for a person. Second, that list is the one Malgieri (2019) found most member states copying. Third, the sceptical reading was upheld by a court, on the old text: in February 2025 the Court of Justice held in Dun and Bradstreet (C-203/22) that Article 15(1)(h) requires a controller to explain the procedure and principles actually applied to reach a specific result. Definitions of meaningful human involvement and of significant effect remain for regulations.

A voluntary framework is left, and a duty is answered for

A team working outside that regime, or ahead of it, reaches for a framework. The National Institute of Standards and Technology's Artificial Intelligence Risk Management Framework (Tabassi 2023) is described as voluntary, rights-preserving, non-sector specific and use-case agnostic. A framework is adopted and can be left. A duty is answered for. That is why Casey and colleagues (2019) predict that algorithmic auditing and design become the operative standard. An audit trail is the form the duty takes when someone asks what happened.

Where the argument is heading

The ICO's guidance is under review, so every summary of the regime, this one included, is provisional. The live research direction is the audit route towards which Malgieri (2019), Edwards and Veale (2017) and Casey (2019) point. Name the duty, name the instrument that discharges it, and keep the evidence. The fifth unit measures what the law leaves untested.

The sources this unit rests on

  • UK Parliament (2018). Data Protection Act 2018. https://www.legislation.gov.uk/ukpga/2018/12/contents Makes provision for the regulation of the processing of information relating to individuals, and carries the UK GDPR's duties into domestic law. The duties themselves sit in the Regulation it accompanies.
  • European Parliament and Council (2016). Regulation (EU) 2016/679. https://eur-lex.europa.eu/eli/reg/2016/679/oj Its Article 5 principles, Articles 13 to 15 information duties and Article 22 automated-decision provisions supply the operational duties, and the United Kingdom retains it as the UK GDPR. Article 22 has since been replaced in that retained version.
  • Information Commissioner's Office (2024). Guide to the UK GDPR and the guide to lawful basis. https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/ States that Article 6 offers seven lawful bases and that consent is only one, and that a basis must be chosen before processing and documented. The page was updated on 2 April 2026 to reflect the Data (Use and Access) Act.
  • Information Commissioner's Office (2024). Data protection impact assessments. https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/accountability-and-governance/data-protection-impact-assessments-dpias/ The design instrument: when an assessment is required, the steps it contains, and when the regulator must be consulted. It adopts the European Data Protection Board's opinion on the ICO's high-risk list.
  • Information Commissioner's Office (2024). Guidance on AI and data protection. https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/artificial-intelligence/guidance-on-ai-and-data-protection/ The regulator's position on automated decisions and fairness, updated on 15 March 2023 after requests from industry. It restates the Article 22 questions a team must ask.
  • UK Parliament (2025). Data (Use and Access) Act 2025. https://www.legislation.gov.uk/ukpga/2025/18/contents Section 80 substitutes Article 22 of the UK GDPR with Articles 22A to 22D, defining a decision as solely automated where there is no meaningful human involvement. It came into force on 19 June 2025 and in full on 5 February 2026.
  • Article 29 Data Protection Working Party (2018). Guidelines on automated decision-making and profiling. https://ec.europa.eu/newsroom/article29/items/612053 These state what counts as a solely automated decision and what safeguards are owed. They are the reading against which the argument over explanation was conducted.
  • Wachter, S., Mittelstadt, B. and Floridi, L. (2017). Why a right to explanation of automated decision-making does not exist in the General Data Protection Regulation. doi:10.1093/idpl/ipx005 Concludes that the Regulation grants a limited right to be informed about the logic of automated decisions, not an explanation of specific decisions. Article 22, it adds, is ambiguous and thinly safeguarded.
  • Selbst, A. D. and Powles, J. (2017). Meaningful information and the right to explanation. doi:10.1093/idpl/ipx022 The reply: Articles 13 to 15 confer rights to meaningful information about the logic involved, which is a right to explanation whether or not the phrase is used. Read functionally, it should let a person exercise the Regulation's other rights.
  • Edwards, L. and Veale, M. (2017). Slave to the algorithm? Why a right to explanation is probably not the remedy you are looking for. doi:10.2139/ssrn.2972855 Argues that a right to explanation is unlikely to remedy the algorithmic harms that drive the demand for it. It locates the better leverage in privacy by design, impact assessment and certification.
  • European Parliament and Council (2024). Regulation (EU) 2024/1689 (Artificial Intelligence Act), Article 86. https://eur-lex.europa.eu/eli/reg/2024/1689/oj Gives a person affected by a decision taken on the output of a high-risk system the right to clear and meaningful explanations of its role. Arnesen (2026), doi:10.1093/idpl/ipag008, reads it against the Court of Justice's ruling in Dun and Bradstreet (C-203/22) that Article 15(1)(h) already requires an explanation of the procedure and principles actually applied, and finds the new right narrower.
  • Malgieri, G. (2019). Automated decision-making in the EU Member States. doi:10.1016/j.clsr.2019.05.002 Found nine member states writing exemptions and safeguards for automated decisions, with scope ranging from legal or similarly detrimental effects to any decision with a significant impact. Most copied the three safeguards; France and Hungary added legibility.
  • Casey, B., Farhangi, A. and Vogl, R. (2019). Rethinking explainable machines: the GDPR's right to explanation debate and the rise of algorithmic audits. doi:10.15779/Z38M32N986 Argues the debate got ahead of itself: the enforcement powers given to supervisory authorities make them the definitive interpreters. Auditing plus design become standard practice.
  • Tabassi, E. (2023). Artificial Intelligence Risk Management Framework (AI RMF 1.0). doi:10.6028/nist.ai.100-1 The voluntary framework released on 26 January 2023, described as rights-preserving, non-sector specific and use-case agnostic. It is the non-statutory route, where a duty is one a team answers for.
  • Rechtbank Den Haag (2020). NJCM and others v the Dutch State (the SyRI judgement), 5 February 2020, case C/09/550982. https://www.rechtspraak.nl/organisatie-en-contact/organisatie/rechtbanken/rechtbank-den-haag/nieuws/2020/02/syri-legislation-in-breach-of-european-convention-on-human-rights Held that the legislation authorising the SyRI welfare fraud risk system breached Article 8 of the European Convention on Human Rights, because its use was insufficiently transparent and verifiable, and declared the provisions to have no binding effect. It is the real failure an impact assessment exists to prevent.
  • van Bekkum, M. and Zuiderveen Borgesius, F. (2021). Digital welfare fraud detection and the Dutch SyRI judgement. European Journal of Social Security. doi:10.1177/13882627211031257 Analyses the judgement, calling it one of the first rulings to invalidate a welfare fraud detection system for breaching the right to privacy, and finds its immediate effects limited because it turns on the circumstances of the case. It keeps the case from being over-read.

Check your understanding & apply it · 4 questions, 1 essay

1A team records that its decision tool is legally safe because the notice states that automated processing is used, describes the main factors, and a review button lets an applicant object. The team reads its position against the Data (Use and Access) Act 2025. Which diagnosis does the unit support?
2Wachter, Mittelstadt and Floridi (2017) and Selbst and Powles (2017) read Articles 13 to 15 of the Regulation and disagree about whether it requires an explanation of an individual decision. Which assessment does the unit support?
3A public body runs a system that combines benefit, tax and housing records to flag households for investigation, and cannot state what the system does to a person or why, as in the SyRI case the District Court of The Hague considered. Which reading does the unit support?
4A team adopts the NIST AI Risk Management Framework and states that it has therefore met its obligations on automated decisions. Which diagnosis does the unit support?
5Essay A product team writes in a launch review: 'Our decision tool is legally safe. The privacy notice tells every applicant that automated processing is used and what the main factors are, and there is a review button if anyone objects. That gives them information about the logic and a route to human intervention, which is everything the data protection regime requires for automated decisions.' Assess this reasoning using the unit's argument. Say what the duties actually oblige, why the distinction between information about a system and an explanation of a decision matters, what an impact assessment should record in this case, and how the Data (Use and Access) Act 2025 changes the test the team is applying. Name the sources you rely on at each step.