Usability as regulated practice: standards, use error and the argument behind a safety case

Healthcare interfaces are not a matter of taste, because someone must show a regulator that use error was addressed. IEC 62366-1 specifies that process, and the FDA states the evidence a submission must carry. Those instruments turn usability into an obligation of documentation, which stops short of the reasoning that connects it to a claim about safety: a validation test that scores a task as completed records performance on the intended path, and the choice of acceptable residual risk stays with the manufacturer.

This eighth of ten units carries the work-system argument into the design process a regulator inspects.

Usability becomes a documented obligation

IEC 62366-1 specifies a process for a manufacturer to analyse, specify, develop and evaluate the usability of a device as it relates to safety, covering the risks of correct use and of use error, that is, normal use (International Electrotechnical Commission, 2020). It can identify but not assess or mitigate abnormal use risks.

ISO 14971 gives use error a place among device risks (International Organization for Standardization, 2019), specifying a process from hazard identification through risk evaluation and control to post-production monitoring. It requires objective criteria for risk acceptability but does not specify acceptable risk levels.

The FDA recommends preliminary analyses, including task analysis, heuristic evaluation and formative evaluation, then validation testing with representative users (US Food and Drug Administration, 2016). The minimum is 15 participants per distinct user population, and a companion guidance states what human factors information a submission must carry (US Food and Drug Administration, 2022).

The safety case, and the argument a test report does not contain

Bishop and Bloomfield define a safety case as a documented body of evidence providing a convincing and valid argument that a system is adequately safe (Bishop and Bloomfield, 1998), decomposed into claims, evidence, the argument linking evidence to claim, and the inference rules that license the link. A validation test produces evidence; the claim that use error has been mitigated so far as is reasonably practicable, and the inference from test performance to it, are separate artefacts.

Leveson and Turner investigated the Therac-25, a medical linear accelerator involved in six massive radiation overdoses between June 1985 and January 1987, from which several patients died (Leveson and Turner, 1993). Their finding is that the overdoses cannot be attributed to a single software defect or operator action, because technical, human, organisational and regulatory factors interacted, so the whole system must be analysed and software is a poor sole barrier. The case predates IEC 62366, so it shows why the argument is worth demanding rather than that the standard fails.

No standard here requires the manufacturer to state those artefacts in a form a third party can attack, so compliance is reachable with a thin, implicit inference: the gap is in the argument, not the evidence.

The guidance expects users, and manufacturers rarely consult them

Money and colleagues interviewed representatives of 11 manufacturers about who they take the intended user to be and who they consult (Money et al., 2011). They found hesitation about formal human factors methods, attributed to barriers to ethical approval, the speed of such activity, and a belief that senior clinicians and champions already know what is needed. The intended user of a home device is often an older patient working alone.

Vincent and colleagues reached the same conclusion from inside development, analysing 19 semi-structured interviews with professionals involved in the design, development and deployment of interactive devices (Vincent, Li and Blandford, 2014). They found barriers to communication with users and between teams, and propose personas and scenarios as mediating representations. The premise of the safety argument, that users are known and represented, is the premise nobody documents.

The methods find problems the file does not record

The inspection methods required do find safety problems. Zhang and colleagues applied a modified heuristic evaluation to two single-channel volumetric infusion pumps, categorising 192 heuristic violations across 89 usability problems in one and 121 across 52 in the other, the first carrying more high-severity problems (Zhang et al., 2003).

Furniss and colleagues observed pump programming in practice across 31 observations and recorded 10 low-level disturbances, which they call unremarkable error (Furniss, Blandford and Mayer, 2011). Sawant and colleagues reviewed usability testing practice across 30 studies and found a field still needing more rigorous procedures (Sawant, Joshi and Sawant, 2023). Those disturbances are the class a validation protocol is least likely to score as failure, since the user completed the task.

What validation testing is designed not to see

The protocol must define in advance the performance that represents success for each task, and participants must represent the intended user population (US Food and Drug Administration, 2016), which includes people who cannot see the screen. A participant who completes a critical task inside the criterion has produced evidence about the intended path.

Gross and colleagues built a tool that predicts the time and mental effort required by operators of a device and application dyad, and applied it to three tasks on a smart insulin pen (Gross et al., 2023).

The contested question: is compliance evidence a safety argument?

One position treats the obligation as adequate because it is external and checkable. IEC 62366-1 requires the manufacturer to analyse, specify, develop and evaluate usability as it relates to safety (IEC, 2020); ISO 14971 requires objective acceptability criteria and post-production monitoring (ISO, 2019); and the FDA expects a pre-specified success criterion and validation testing with at least 15 participants per user population (FDA, 2016). On this reading, what Money and Vincent describe is a problem of enforcement, not of the standard.

The second position holds that the standard sets the evidence and not the reasoning. Nothing in IEC 62366-1 requires the manufacturer to state the claim the evidence supports, or to make the inference explicit and falsifiable, which a safety case demands (Bishop and Bloomfield, 1998). ISO 14971 returns the acceptability threshold to the manufacturer. The scope excludes abnormal use (IEC, 2020), while the field observations that matter are of ordinary practice (Furniss, Blandford and Mayer, 2011), and the industry's own complexity instrument assumes error-free performance (Gross et al., 2023). The strongest counter-argument is that this position is about enforcement dressed as an argument about the standard, because no study in this unit measured whether compliance reduced harm.

A third position says the framework does not transfer to devices that learn or recommend. Habli and colleagues argue that none of the actors in their model of clinical artificial intelligence robustly fulfil the traditional conditions of moral accountability for its decisions. They propose moving from static to dynamic assurance (Habli, Lawton and Porter, 2020). Sujan and colleagues note the lack of independent evaluation in use and argue that human factors research should accompany development (Sujan et al., 2019). Regulation has since caught up in part: the Artificial Intelligence Act makes effective human oversight of high-risk medical device software a legal obligation, applying from August 2027 (European Parliament and Council of the European Union, 2024).

What would settle it is a comparison, not another position statement: device families of comparable risk compared on post-market use-error reports, one carrying an explicit claim-evidence argument with its residual risks stated and the other a validation report alone. If the argument matters, the falsifiable version should name its residual risks in advance and the report-only version should be surprised by the errors it later records.

What to do while the dispute is open

The evidence on the two sides is unequal. That compliance is adequate rests on standards that are checkable and on inspection methods that find real problems. That the argument is missing rests on a safety-case framework and on observations of practice, not on an outcome. The strongest objection to this unit's thesis is that no study here measured whether compliance reduced harm, so the case for an explicit argument is itself untested. While the dispute is open, a designer should write the claim and the evidence, name the inference that links them, state the residual risks, and leave the argument in a form a third party can attack.

Where this is heading

Habli and colleagues argue for dynamic assurance, in which safety is revisited after deployment rather than settled in design (Habli, Lawton and Porter, 2020), and Sujan and colleagues place the human factors work in the same timeline, with research accompanying development and independent evaluation in use (Sujan et al., 2019). The open question is whether the argument, rather than the evidence, becomes what the regulator inspects.

Sources

• Bishop, P. and Bloomfield, R. (1998). A Methodology for Safety Case Development. Industrial Perspectives of Safety-critical Systems. doi:10.1007/978-1-4471-1534-2_14 Defines a safety case as a documented body of evidence providing a convincing and valid argument that a system is adequately safe, decomposed into claims, evidence, argument and inference. It names the parts a usability file omits.
• International Electrotechnical Commission (2020). IEC 62366-1:2015+AMD1:2020 Medical devices - Part 1: Application of usability engineering to medical devices. International Electrotechnical Commission. https://webstore.iec.ch/en/publication/67220 Specifies a process to analyse, specify, develop and evaluate device usability as it relates to safety and to mitigate the risks of correct use and use error. Its scope excludes abnormal use, the boundary this unit turns on.
• International Organization for Standardization (2019). ISO 14971:2019 Medical devices - Application of risk management to medical devices. International Organization for Standardization. https://www.iso.org/standard/72704.html Specifies risk management for devices across the life cycle, from hazard identification to post-production monitoring. It requires objective criteria for risk acceptability but declines to set acceptable risk levels, which is where the manufacturer's judgement enters the argument.
• US Food and Drug Administration (2016). Applying Human Factors and Usability Engineering to Medical Devices: Guidance for Industry and Food and Drug Administration Staff. US Food and Drug Administration. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/applying-human-factors-and-usability-engineering-medical-devices Recommends preliminary analyses including task analysis, heuristic evaluation and formative evaluation, then validation testing with representative users. It sets a minimum of 15 participants per user population, a countable obligation on participation rather than a stated claim.
• US Food and Drug Administration (2022). Content of Human Factors Information in Medical Device Marketing Submissions: Guidance for Industry and Food and Drug Administration Staff. US Food and Drug Administration. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/content-human-factors-information-medical-device-marketing-submissions Sets out the risk-based framework for the human factors information a premarket submission must document, as a companion to the 2016 guidance. It turns the usability obligation into a specified content and format.
• Money, A. G., Barnett, J., Kuljis, J., Craven, M. P., Martin, J. L. and Young, T. (2011). The role of the user within the medical device design and development process: medical device manufacturers' perspectives. BMC Medical Informatics and Decision Making. doi:10.1186/1472-6947-11-15 Reports interviews with representatives of 11 manufacturers about their intended user. It finds hesitation about formal human factors methods, attributed to ethical-approval barriers and a belief that senior clinicians already know what is needed, so the users the argument depends on may never be consulted.
• Vincent, C. J., Li, Y. and Blandford, A. (2014). Integration of human factors and ergonomics during medical device design and development: It's all about communication. Applied Ergonomics. doi:10.1016/j.apergo.2013.05.009 Reports a thematic analysis of 19 semi-structured interviews with professionals who design and deploy interactive devices. The barriers to safety were barriers to communication, and the authors propose personas and scenarios as a remedy.
• Zhang, J., Johnson, T. R., Patel, V. L., Paige, D. L. and Kubose, T. (2003). Using usability heuristics to evaluate patient safety of medical devices. Journal of Biomedical Informatics. doi:10.1016/s1532-0464(03)00060-1 Applied a modified heuristic evaluation to two volumetric infusion pumps. It categorised 192 heuristic violations across 89 usability problems in one pump and 121 across 52 in the other, showing the inspection layer doing safety work before a design is frozen.
• Furniss, D., Blandford, A. and Mayer, A. (2011). Unremarkable errors: low-level disturbances in infusion pump use. Electronic Workshops in Computing. doi:10.14236/ewic/hci2011.47 Reports an observational study of pump programming in practice, recording 10 low-level disturbances across 31 observations, which they call unremarkable errors. These are the disturbances a validation protocol is least likely to record as failure.
• Sawant, S., Joshi, P. and Sawant, S. (2023). Usability testing of Healthcare Devices: A review of the current UX methods used for usability testing of healthcare devices. Proceedings of the Human Factors and Ergonomics Society Annual Meeting. doi:10.1177/21695067231192409 Reviews device usability testing practice across 30 studies, and finds a field still needing more rigorous procedures.
• Gross, M., Roberts, C., Stinson, K. and Wiltman, S. (2023). Improving medical device usability by reducing complexity using a novel predictive models-based user interface assessment tool. Human Factors in Healthcare. doi:10.1016/j.hfh.2023.100041 Predicts with a spreadsheet tool the time and mental effort required by operators of a device and application dyad, tested on a smart insulin pen. It assumes the patient follows the intended path, and supplements rather than replaces usability testing.
• Sujan, M., Furniss, D., Grundy, K., et al. (2019). Human factors challenges for the safe use of artificial intelligence in patient care. BMJ Health & Care Informatics. doi:10.1136/bmjhci-2019-100081 Argues that the human factors challenges of clinical artificial intelligence arise where humans and systems meet, in automation bias, handover and situation awareness. It reports a lack of independent evaluation in use.
• Habli, I., Lawton, T. and Porter, Z. (2020). Artificial intelligence in health care: accountability and safety. Bulletin of the World Health Organization. doi:10.2471/blt.19.237487 Argues that clinical artificial intelligence challenges the practices of assigning blame and assuring safety, and that none of the actors in their model robustly fulfil the conditions of moral accountability. It proposes dynamic rather than static assurance.
• Leveson, N. G. and Turner, C. S. (1993). An investigation of the Therac-25 accidents. Computer. doi:10.1109/mc.1993.274940 Investigated the Therac-25 medical linear accelerator, involved in six massive radiation overdoses between June 1985 and January 1987 from which several patients died. It attributes the overdoses to interacting technical, human, organisational and regulatory factors rather than a single defect or action.
• European Parliament and Council of the European Union (2024). Regulation (EU) 2024/1689 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act). Official Journal of the European Union. https://eur-lex.europa.eu/eli/reg/2024/1689/oj Classifies artificial intelligence that is a safety component of a medical device as high risk, and makes effective human oversight a legal obligation, so oversight becomes something the manufacturer must document rather than an argument it may omit. The provisions apply from August 2027.