Failure is not additive: probability, expected value and the independence assumption

Year 1 / Module 3 Mathematics for interaction / Unit 5 / Failure is not additive: probability, expected value and the independence assumption

Failure is not additive: probability, expected value and the independence assumption

HCI1103 · Unit 5 of 10

Failure rates add only when events are disjoint and multiply only when they are independent, so almost every availability figure quoted for an interactive system is an arithmetic artefact of an assumption its own components violate.

Failure is not additive: probability, expected value and the independence assumption

An availability figure is usually produced in one sentence. A flow has three steps, each failing about once in fifty attempts. So it succeeds 0.98 times 0.98 times 0.98 of the time, or 94 per cent. The multiplication is correct. The claim is not, because the multiplication is licensed by an assumption the sentence never states. Interface components violate that assumption as a rule rather than as an exception (Knight and Leveson, 1986).

This unit argues that failure rates combine under two rules, and that each rule carries a precondition. The precondition is an empirical claim about the parts rather than about the arithmetic. The literature has tested that precondition and begun to supply its replacement. Since the inputs are themselves uncertain estimates, the honest deliverable is the dependence structure that was assumed, with a range on how far the answer moves when it is relaxed. Unit 6 opens the block of laws measured on people rather than on artefacts.

Two rules, two preconditions, and an answer that hides which was used

Feller (1968) treats independence as a modelling assumption rather than a property of the world, and is explicit that the multiplication rule holds only for independent events. Grinstead and Snell (2003) give the companion rule: probabilities of a union are added and the overlap subtracted.

Add the rates when the events cannot both happen, and subtract the overlap when they can (Grinstead and Snell, 2003).
Multiply the rates when the events are independent, since only then is the product the joint probability; otherwise it can err in either direction (Feller, 1968).

Two steps each fail with probability 0.02. If the failures are independent, the chance that at least one fails is 1 minus 0.98 squared, or 0.0396, while adding the rates gives 0.04. The difference of 0.0004 is the case in which both fail, which the sum counts twice. The expected number of attempts to one clean pass is 1 divided by 0.9604, or 1.041, the mean of the geometric distribution (Grinstead and Snell, 2003).

The numbers being multiplied are estimates with wide uncertainty

Swain and Guttmann (1983) assembled the handbook of human reliability analysis for the United States Nuclear Regulatory Commission, whose tables and human error rate prediction technique are the reference values for many later estimates.

A concrete instance shows how the tables get used. A procedure has three operator steps, each carrying a handbook error probability of 0.01, and a risk assessment multiplies them to report a per-task failure probability of one in a million. That multiplication is licensed by the independence of the three steps, which the handbook does not supply, and by the constancy of the values, which the tables present as expert central estimates.

French, Bedford, Pollard and Soane (2011) review the methods as critics, arguing that they rest on assumptions rarely justified: they name the decomposability of tasks and the independence of failure modes. The multiplication assumes the multiplicands are constants, when the literature supplying them says they are not.

A failure rate is not a property of the step

Swets (1973) sets out the receiver operating characteristic, a technique that isolates the observer's decision criterion from the acuity of the discrimination. The same discrimination therefore yields very different hit and false alarm rates: a monitoring screen that raises an alert has a sensitivity and a criterion, and its false alarm rate is a decision about how much noise an operator tolerates rather than a fixed constant. Multiplying that rate by a second assumes both are stable under the conditions of the combination, which is what the threshold account denies. The rate is not a property of the population either. The handbook's values are central estimates from expert judgement on a particular operator population, and Swets makes the criterion a decision that varies with the person and the conditions. Where those who use the interface are not those the rate was estimated on, the multiplicand is less constant than the product assumes.

Independence was tested, and it failed

The precondition behind every product of failure rates was tested directly in software engineering. N-version programming runs independently written programs in parallel and trusts their majority, so its reliability gain depends on the versions failing independently. Knight and Leveson (1986) had 27 versions written from one specification at two universities and subjected them to one million tests: individually extremely reliable, but with substantially more tests failing in more than one version than independence predicted.

The clearest field case is a network rather than a program. On 15 January 1990 a minor hardware fault in one of AT&T's 4ESS toll switches triggered its recovery software, and the failures repeated across all 114 switches in the network for approaching half a day, so long-distance calls became almost impossible to complete (Neumann, 2006). Every switch ran the same software, and one defective statement in the recovery code made the recovery mechanism the propagation path. A product of failure rates cannot represent that: 114 individually reliable components failed together because the dependence was designed in. The case does not show that replication is useless, only that its benefit depends on a diversity the design has to supply.

Eckhardt and Lee (1985) had given the mathematics of what replaces it. Their framework centres on an intensity function describing the propensity of programmers to introduce faults that make components fail together, from which they derive a condition under which a multiversion system beats a single version. Littlewood and Miller (1989) generalise the model to versions built under diverse methodologies. Diversity of decision implies a likely diversity of behaviour, so it decreases the probability of simultaneous failure, and better-than-independent failure is theoretically possible.

Why the wrong combination is written so easily

Casscells, Schoenberger and Graboys (1978) put a positive test result to 60 house officers, medical students and attending physicians at four teaching hospitals, with a stated prevalence and false positive rate. Fewer than one in five gave the correct posterior probability, and most answers sat close to the false positive rate. Eddy (1982) reports the same inversion for mammography, with physicians overestimating the probability of cancer given a positive result partly by inverting the conditional they were given.

Tversky and Kahneman (1974) name the mechanism. Judgement under uncertainty proceeds by heuristics that are usually effective and produce systematic, predictable errors. The multiplication rule is available and the conditioning question is not, so the product gets written.

Contested question: where the error lives, and what dependence is for

The first disagreement is where the error lives. Casscells and colleagues (1978), Eddy (1982) and Tversky and Kahneman (1974) treat the error as a property of the reasoner. Gigerenzer and Hoffrage (1995) hold that the format decides whether the algorithm can be run at all: Bayesian algorithms are computationally simpler in frequency formats, and statistically naive participants derived up to 50 per cent of their inferences by Bayesian algorithms once the data were stated as frequencies. On that account the correction is an instruction about how an estimate is written. The two sides differ in the kind of evidence they carry. The reasoner side rests on a classic demonstration with 60 clinicians at four teaching hospitals and on a heuristics literature replicated widely; the representation side rests on an analysis of several thousand written solutions, which shows that the format changes the answer but not that a trained engineer in the field would have run the algorithm. What would settle it is a trial giving the same failure data, in both formats, to the people who write the estimate, and measuring the posterior they reach.

The second disagreement is what dependence is for. Knight and Leveson (1986) refute independence as a description, which makes dependence an obstacle to any product of rates. Eckhardt and Lee (1985) and Littlewood and Miller (1989) treat the same dependence as a structured quantity whose parameters design can act on. French and colleagues (2011) accept that the methods produce numbers, while arguing that the assumptions behind them are rarely justified. Podofillini and Dang (2025) give that critique an empirical floor, reading six operational events for the coupling factors that make human failure events dependent and testing a widely used dependence decision tree against them. Zhao and colleagues (2022) place several methods in one Bayesian framework and compare their estimates against human performance data. What would settle it is a preregistered study reporting joint failure frequencies for real interface components. It would estimate the dependence parameters rather than assume them, and show how far the system-level figure moves across that range. An availability figure quoted without its dependence assumption is a claim about the arithmetic, not the system. Meanwhile, put the dependence assumption beside every combined figure, give the range the answer takes when it is relaxed, and present failure data as frequencies to the people who will combine them.

Where the argument is heading

The direction of travel is to make dependence a reported quantity and the estimate a range. Zhao and colleagues (2022) measure competing methods against performance data rather than against one another's assumptions, and Podofillini and Dang (2025) test one of those methods against six operational events, finding that it holds for most cases but certifies independence in some where the event narrative does not support it. Gigerenzer and Hoffrage (1995) point at the other half. A format that lets the algorithm run is a design decision about the estimate. The study still lacking would measure the dependence structure of a real interface and report the joint failure frequencies, because independence has been refuted where it was tested carefully and merely asserted everywhere else.

The sources this unit rests on

Neumann, P. G. (2006). System and network trustworthiness in perspective. Proceedings of the 13th ACM Conference on Computer and Communications Security (CCS). https://www.csl.sri.com/~neumann/ccs06.pdf Describes the 1990 collapse of AT&T's long-distance network, when an untested upgrade to the recovery software in 114 4ESS switches made one node's crash propagate across the network for most of half a day.
Feller, W. (1968). An Introduction to Probability Theory and Its Applications, Volume 1, 3rd edition. Wiley. https://archive.org/details/introductiontopr0000fell Treats independence as a modelling assumption rather than a property of the world: the multiplication rule holds for independent events only.
Grinstead, C. M. and Snell, J. L. (2003). Introduction to Probability. American Mathematical Society, freely redistributable electronic edition. https://math.dartmouth.edu/~prob/prob/prob.pdf Defines the geometric distribution, shows its mean to be one over the success probability, and gives the addition rule with its overlap correction.
Tversky, A. and Kahneman, D. (1974). Judgement under uncertainty: heuristics and biases. Science, 185(4157), 1124-1131. doi:10.1126/science.185.4157.1124 Describes three heuristics used in judgement under uncertainty, economical and usually effective but producing systematic errors, including neglect of prior probability.
Casscells, W., Schoenberger, A. and Graboys, T. B. (1978). Interpretation by physicians of clinical laboratory results. New England Journal of Medicine, 299(18), 999-1001. doi:10.1056/NEJM197811022991808 Put a positive test result, with a stated prevalence and false positive rate, to 60 clinicians and students at four teaching hospitals: fewer than one in five gave the correct posterior probability.
Eddy, D. M. (1982). Probabilistic reasoning in clinical medicine: problems and opportunities. In Judgement under Uncertainty: Heuristics and Biases, Cambridge University Press, 249-267. doi:10.1017/CBO9780511809477.019 Finds that physicians reading a mammogram overestimate the probability of cancer given a positive test, partly by inverting the conditional they were given, so the failure precedes the multiplication.
Gigerenzer, G. and Hoffrage, U. (1995). How to improve Bayesian reasoning without instruction: frequency formats. Psychological Review, 102(4), 684-704. doi:10.1037/0033-295X.102.4.684 Show Bayesian algorithms to be computationally simpler in frequency formats than in probability formats, and analyse several thousand solutions: naive participants derived up to 50 per cent of their inferences by Bayesian algorithms once the data were frequencies.
Knight, J. C. and Leveson, N. G. (1986). An experimental evaluation of the assumption of independence in multiversion programming. IEEE Transactions on Software Engineering, SE-12(1), 96-109. doi:10.1109/TSE.1986.6312924 Had 27 versions written independently from one specification at two universities and subjected to one million tests: individually reliable, but with far more tests failing in more than one version than independence predicted.
Eckhardt, D. E. and Lee, L. D. (1985). A theoretical basis for the analysis of multiversion software subject to coincident errors. IEEE Transactions on Software Engineering, SE-11(12), 1511-1517. doi:10.1109/TSE.1985.231895 Develops a framework centred on an intensity function describing the propensity of programmers to introduce faults that make components fail together, and derives when a multiversion system beats a single version.
Littlewood, B. and Miller, D. R. (1989). Conceptual modelling of coincident failures in multiversion software. IEEE Transactions on Software Engineering, 15(12), 1596-1614. doi:10.1109/32.58771 Models coincident failures through a duality between input choice and program choice, generalised to versions built under diverse methodologies: diversity decreases the probability of simultaneous failure, and better-than-independent failure is theoretically possible.
Swain, A. D. and Guttmann, H. E. (1983). Handbook of human reliability analysis with emphasis on nuclear power plant applications. US Nuclear Regulatory Commission, NUREG/CR-1278. doi:10.2172/5752058 Presents methods and estimated human error probabilities for operator error in nuclear power plants as part of probabilistic risk assessment; its tables are the reference values for later estimates.
French, S., Bedford, T., Pollard, S. J. T. and Soane, E. (2011). Human reliability analysis: a critique and review for managers. Safety Science, 49(6), 753-763. doi:10.1016/j.ssci.2011.02.008 Reviews human reliability analysis and argues that the methods rest on assumptions rarely justified, naming the decomposability of tasks and the independence of failure modes.
Swets, J. A. (1973). The relative operating characteristic in psychology. Science, 182(4116), 990-1000. doi:10.1126/science.182.4116.990 Sets out the receiver operating characteristic as a technique isolating the observer's decision criterion from the acuity of the discrimination, so the same discrimination yields very different hit and false alarm rates depending on where the criterion sits.
Zhao, Y. and others (2022). A Bayesian approach to comparing human reliability analysis methods using human performance data. Reliability Engineering and System Safety, 219, 108213. doi:10.1016/j.ress.2021.108213 Places several human reliability analysis methods in one Bayesian framework and compares their estimates against human performance data.
Podofillini, L. and Dang, V. N. (2025). Dependence in human reliability analysis and in operational events. Progress in Nuclear Energy, 190, 105991. doi:10.1016/j.pnucene.2025.105991 Reads six operational events and identifies the coupling factors that make human failure events dependent, then tests a widely used dependence decision tree against them. The method is practical, but it can certify independence where the event narrative does not support it.

Check your understanding & apply it · 4 questions, 1 essay

1Two steps in a flow each fail with probability 0.02. What do the two combination rules give for the chance that at least one step fails, and what does the difference between them amount to?

2A risk report multiplies three operator step failure probabilities and quotes a per-task figure. On the unit's evidence, which finding most directly undermines the independence the multiplication requires?

3A team's estimate is written as probabilities and the engineers keep inverting the conditional. What does Gigerenzer and Hoffrage's (1995) evidence suggest the team should change?

4The unit argues that the failure rate of a step is not a fixed constant that can simply be multiplied by another. Which reason does it give?

5Essay A payments team reports that its checkout flow succeeds 94 per cent of the time, having multiplied the success probability of each of three steps, each quoted as failing about once in fifty attempts. Using Feller (1968) and Grinstead and Snell (2003) on the two combination rules and their preconditions, Swain and Guttmann (1983) and French and colleagues (2011) on where such component numbers come from, Knight and Leveson (1986), Eckhardt and Lee (1985) and Littlewood and Miller (1989) on the independence assumption, Swets (1973) on decision criteria, and Casscells and colleagues (1978), Eddy (1982), Tversky and Kahneman (1974) and Gigerenzer and Hoffrage (1995) on how the combination is reasoned about, write an assessment that (a) states which rule the team used and which precondition it requires, and what the figure would be under the other rule, (b) says what the three component numbers are and what that implies for the precision of the product, (c) sets out the strongest case for treating dependence as a design variable rather than an obstacle, and the strongest case against the team's habit of quoting a single figure, and (d) specifies a study that would settle the disagreement, naming the observations it would make and the result that would count against the team's figure.

The team has multiplied three success probabilities, so it has used the multiplication rule, which Feller (1968) licenses only for independent events. If the three step failures were instead treated as disjoint, the probabilities of failure would add rather than multiply, and the unit's two-step case shows how far apart the two answers are: 0.04 by addition against 0.0396 under independence, a difference equal to the chance that both steps fail, which the sum counts twice (Grinstead and Snell, 2003). Neither rule is more accurate as arithmetic; the rules differ in what they assume, and the report does not say which assumption it made. The three component figures are human error probabilities of the kind tabulated by Swain and Guttmann (1983) for probabilistic risk assessment, and French and colleagues (2011) argue that such numbers rest on assumptions that are rarely justified, including the decomposability of tasks and the independence of failure modes, so the product inherits the uncertainty of three estimates and adds an assumption. Swets (1973) removes a further constant from the calculation, since a step's error rate partly reflects where its decision criterion was placed, so it is a design choice rather than a measurement. The strongest case for treating dependence as a design variable is Littlewood and Miller (1989), who show that diverse methodologies decrease the probability of simultaneous failure and that better than independent behaviour is theoretically possible, with Eckhardt and Lee (1985) supplying the intensity function that makes the effect a parameter rather than a hope. The strongest case against the single figure is Knight and Leveson (1986), whose 27 versions failed together substantially more often than independence predicted, which is what makes the report's silence about dependence a defect rather than an omission. The reasoning literature widens the objection: Casscells and colleagues (1978) and Eddy (1982) show experts answering the wrong conditional, and Gigerenzer and Hoffrage (1995) show the answer improving when the same data are stated as frequencies, so the correction is partly a matter of how the estimate is written. A study that would settle it would pre-register a set of checkout steps, log joint failures across the whole flow rather than per step, estimate the dependence between steps rather than assume it, and report how far the flow-level figure moves across the plausible range of dependence, using frequency counts as Gigerenzer and Hoffrage recommend. If the joint failure count matches the product within sampling error, the team's figure survives; if steps fail together more often, the 94 per cent is an artefact of the arithmetic rather than a property of the flow.