Attrition: the users who leave are the finding

Year 4 / Module 29 In-the-wild and longitudinal deployment studies / Unit 4 / Attrition: the users who leave are the finding

HCI4105 · Unit 4 of 10

Attrition is the defining feature of a deployed intervention rather than a defect of one study, so a deployment should model who leaves and why instead of treating dropout as missing data.

Attrition: the users who leave are the finding

The most reliable result a deployment produces is that most people stop: sixty people install the app, and by the third month the log shows twelve. The trial tradition reads that as damage and reports completion among those who stayed.

This unit argues the reverse. This fourth of ten units turns the same question round: the three before it asked what a deployment shows, and this one asks who the deployment keeps. Attrition is the defining feature of a deployed intervention, not a defect of one study. The users who leave are the finding. The work is to model who leaves and why, rather than treat dropout as missing data.

The failure that turned out to be the finding

Eysenbach named the pattern and reversed its meaning. The law of attrition states that a substantial proportion of users drop out of any eHealth trial before completion, or stop using the application. That attrition is a characteristic of the medium, not a defect of one study (Eysenbach, 2005), and high dropout can be typical for internet self-help. Usage metrics, the determinants of attrition and the characteristics of those who stay all belong in the report. The method follows. It uses survival analysis of attrition curves and a usage half-life. It treats differential dropout as a metric in its own right. It warns that with high dropout, intention-to-treat efficacy measures underestimate the impact on those who continue (Eysenbach, 2005).

The pooled estimates exist, and they disagree

Torous and colleagues meta-analysed 18 randomised trials of smartphone apps for depressive symptoms, with 3,336 participants. They found pooled dropout of 26.2 per cent, rising to 47.8 per cent after adjustment for publication bias (Torous et al., 2020). Linardon and Fuller-Tyszkiewicz put mean attrition across 70 randomised trials at 24.1 per cent at short-term follow-up. That figure rises to 35.5 per cent at longer-term follow-up (Linardon and Fuller-Tyszkiewicz, 2020). Meyerowitz-Katz and colleagues meta-analysed 17 chronic disease app studies. They reported pooled dropout of 43 per cent, with 49 per cent in observational studies against 40 per cent in trials. Heterogeneity was above 99 per cent (Meyerowitz-Katz et al., 2020). A pooled number whose studies disagree by more than chance can describe a literature, not yet a medium.

The newest and largest synthesis separates the three quantities. Liu and colleagues pooled 79 randomised trials of depression and anxiety apps: uptake was 92.4 per cent, adherence 61.8 per cent, and posttest attrition 18.6 per cent, rising to 28.4 per cent at follow-up (Liu et al., 2026).

The trial number is not the deployment number

Baumel and colleagues separated the programme from the study of it. They paired published engagement figures for off-the-shelf unguided mental health programmes with objective real-world usage of the same programmes. The comparison covered 13 papers and 14 comparable usage metrics (Baumel et al., 2019). Three papers on lay users with no proactive trial procedures gave ratios of 0.84, 1.05 and 1.27. In the 11 comparisons from studies that proactively recruited, median reported usage was 4.06 times real-world usage, with an interquartile range of 4.49. The gap belongs to the setting.

Fleming and colleagues found the same fragmentation from the service side. Across 10 papers on 7 publicly available programmes, completion or sustained use ran from 0.5 to 28.6 per cent, so inconsistent that they ask for routine reporting of implementation data (Fleming et al., 2018). Kelders and colleagues add where the number comes from. Across 101 articles on 83 interventions, about 50 per cent of participants adhere, and randomised rather than observational design predicted better adherence (Kelders et al., 2012). Design moves the number. So does being studied.

What predicts leaving keeps moving to the design, and the person-level predictors do not survive

Pratap and colleagues pooled eight remote digital health studies run between 2014 and 2019, in which more than 100,000 participants completed nearly 3.5 million evaluations. Median retention ranged from 2 to 26 days, with a median of 5.5. Clinician referral was associated with 40 additional days, compensation with 22, having the clinical condition with 7 and older age with 4 (Pratap et al., 2020). Retention is partly bought, and a service that cannot afford the compensation will lose those days. Weeks, not months, is the honest baseline.

Geraghty and colleagues supply the cleanest natural experiment, two identical cohorts of an international smoking cessation intervention. One was fully automated, with 16,430 participants; the other added live contact for initial non-responders, with 1,000. Attrition was 48.1 and 10.8 per cent (Geraghty et al., 2013). Dependency and education predicted attrition under automation, while younger age was the sole predictor under live contact. The meta-analyses report the same levers: feedback, reminders and compensation lower attrition, and online enrolment raises it (Torous et al., 2020; Linardon and Fuller-Tyszkiewicz, 2020). Liu and colleagues add one that points the other way: attrition was lower in apps without gamification features (Liu et al., 2026).

If attrition were a property of the person, baseline characteristics would predict it. They largely do not. Beatty and Binnion reviewed 36 studies. Female gender, higher treatment expectancy, sufficient time and personalised content predicted higher adherence. Age, symptom severity and control group allocation gave mixed findings, and most variables predicted nothing (Beatty and Binnion, 2016). Linardon and Fuller-Tyszkiewicz state it more starkly: no participant-level baseline characteristic reliably predicted attrition (Linardon and Fuller-Tyszkiewicz, 2020).

What is lost when the analysis ignores who left

The omission has a price, though a modest one. Linardon and Fuller-Tyszkiewicz found evidence of attrition bias in the many trials that reported no intention-to-treat analysis. Where trials reported both analyses, the mean difference in the effect size on primary outcomes was only 0.18 (Linardon and Fuller-Tyszkiewicz, 2020). Eysenbach's warning is the mirror of it: a high-dropout trial understates the effect on those who continue (Eysenbach, 2005). The analysis choice moves the estimate modestly. It does not tell you who left.

Donkin and colleagues show how rarely the underlying question is asked. Across 69 e-therapy studies that reported an adherence measure, only 33 examined the relationship between adherence and outcome. The measure that related to outcome differed by target: logins in physical health interventions and module completion in psychological ones (Donkin et al., 2011). The relation between use and benefit was therefore untested in half the studies that recorded use. The association is correlational across heterogeneous studies, so it cannot say whether adherence caused the change or the change caused the adherence.

Contested question: is attrition a property of the person or of the study?

Three positions are on the table. The first locates attrition in the person. Beatty and Binnion name gender, treatment expectancy and available time (Beatty and Binnion, 2016). Christensen and colleagues name disease severity and treatment length (Christensen et al., 2009). The second locates it in the study. Baumel and colleagues measured a fourfold gap between reported and real-world usage of the same programmes (Baumel et al., 2019). Pratap and colleagues attribute tens of days of retention to referral and compensation (Pratap et al., 2020).

The third position concerns the measurement. Meyerowitz-Katz and colleagues report pooled dropout of 43 per cent with heterogeneity above 99 per cent and inconsistent measurement (Meyerowitz-Katz et al., 2020). If leaving is defined differently in each report, who leaves depends on what leaving was taken to mean. The finding of Kelders and colleagues that trial status predicts adherence belongs here too (Kelders et al., 2012).

The person-level one rests on systematic reviews of predictors (Beatty and Binnion, 2016); the study-level one on a paired comparison of reported and objective usage and a pooled analysis of eight studies (Baumel et al., 2019; Pratap et al., 2020); the measurement one on a meta-analysis with heterogeneity above 99 per cent (Meyerowitz-Katz et al., 2020). What would settle the first two is the decisive version of the Baumel comparison, which has not been run: one programme, one definition of usage, one follow-up window, inside a trial and in routine service at once, with the recruitment channel held constant. While the dispute is open, report attrition per programme with its definition and channel stated. The strongest objection is that a person-level model would find the arrangement through the person, so the two are not separable.

Where the area is heading

Two shifts are visible. Attrition is becoming a modelled dependent variable. Reips, Overlander and Bannert turn the practice into a method and a tool: dropout and non-response in internet-based research are dependent variables, they argue, and their dropR package plots the curves, with chi-square values, odds ratios and Kaplan-Meier survival statistics (Reips et al., 2025). Two decades after Eysenbach asked for survival analysis of attrition, the routine instrument exists, and what remains scarce is the reason a particular person stopped, which no curve supplies.

Retention is becoming a design property, with a value in days placed on referral and compensation (Pratap et al., 2020) and a demand for routine implementation reporting (Fleming et al., 2018). The open question is whether shared definitions make the pooled estimates converge, or whether attrition stays a per-programme number.

The sources this unit rests on

Each note states what the cited document did and found. Titles are in British spelling, and identifiers were checked against registered metadata on 2 November 2026.

Eysenbach, G. (2005). The law of attrition. doi:10.2196/jmir.7.1.e11 Names the law of attrition: a substantial proportion of users drop out of any eHealth trial, and asks for survival analysis, a usage half-life and the characteristics of those who stay.
Christensen, H., Griffiths, K. M. and Farrer, L. (2009). Adherence in internet interventions for anxiety and depression. doi:10.2196/jmir.1194 A systematic review that named disease severity and treatment length among the predictors of adherence. Its trial attrition range, about 1 to 50 per cent, predates the pooled estimates used above.
Kelders, S. M. et al. (2012). Persuasive system design does matter: a systematic review of adherence to web-based interventions. doi:10.2196/jmir.2104 Codes persuasive technology elements and adherence across 101 articles on 83 interventions. About 50 per cent of participants adhered, and more counsellor interaction and randomised rather than observational design predicted better adherence.
Geraghty, A. W. A. et al. (2013). Understanding attrition from international internet health interventions: a step towards global eHealth. doi:10.1093/heapro/das029 Models attrition in two identical cohorts of an international smoking cessation intervention, one fully automated (16,430 participants) and one with live contact (1,000). Attrition was 48.1 and 10.8 per cent, with younger age the only predictor under live contact.
Beatty, L. and Binnion, C. (2016). A systematic review of predictors of, and reasons for, adherence to online psychological interventions. doi:10.1007/s12529-016-9556-9 Reviews 36 studies of predictors of adherence. Female gender, higher treatment expectancy, sufficient time and personalised content predicted higher adherence, while most other variables, including age and symptom severity, predicted nothing.
Fleming, T. et al. (2018). Beyond the trial: systematic review of real-world uptake and engagement with digital self-help interventions for depression, low mood, or anxiety. doi:10.2196/jmir.9275 Screens 970 records and includes 10 papers on 7 implemented self-help programmes. Completion or sustained use ran from 0.5 to 28.6 per cent, with little consistency in measures, so the authors ask for implementation data to be reported routinely.
Baumel, A., Edan, S. and Kane, J. M. (2019). Is there a trial bias impacting user engagement with unguided e-mental health interventions? A systematic comparison of published reports and real-world usage of the same programmes. doi:10.1093/tbm/ibz147 Pairs published engagement figures with objective real-world usage of the same programmes, across 13 papers and 14 comparable metrics. Three papers on lay users gave ratios of 0.84, 1.05 and 1.27; the 11 comparisons from recruited studies reported a median usage rate 4.06 times real-world usage.
Torous, J. et al. (2020). Dropout rates in clinical trials of smartphone apps for depressive symptoms: a systematic review and meta-analysis. doi:10.1016/j.jad.2019.11.167 Meta-analyses 18 randomised trials, 3,336 participants. Pooled dropout was 26.2 per cent, rising to 47.8 per cent after adjustment for publication bias, and lower where studies offered human feedback or in-app mood monitoring.
Meyerowitz-Katz, G. et al. (2020). Rates of attrition and dropout in app-based interventions for chronic disease: systematic review and meta-analysis. doi:10.2196/20283 Meta-analyses 17 app studies for chronic disease, 9 randomised and 8 observational. Pooled dropout was 43 per cent, higher in observational studies at 49 per cent than in trials at 40 per cent, with heterogeneity above 99 per cent.
Linardon, J. and Fuller-Tyszkiewicz, M. (2020). Attrition and adherence in smartphone-delivered interventions for mental health problems: a systematic and meta-analytic review. doi:10.1037/ccp0000459 Reviews 70 randomised trials of smartphone mental health interventions. Mean attrition was 24.1 per cent at short-term and 35.5 per cent at longer-term follow-up; no participant-level baseline characteristic reliably predicted attrition, and the gap between per-protocol and intention-to-treat effect sizes was only 0.18.
Liu, C., Torous, J. and Linardon, J. (2026). Uptake, adherence, and attrition in clinical trials of depression and anxiety apps: a systematic review and meta-analysis. doi:10.1001/jamapsychiatry.2025.3439 Meta-analyses 79 randomised trials of depression and anxiety apps, 92 app conditions and 78 controls. Uptake was 92.4 per cent and adherence 61.8 per cent, posttest attrition 18.6 per cent rising to 28.4 per cent at follow-up, and lower with reminders or human contact and without gamification.
Pratap, A. et al. (2020). Indicators of retention in remote digital health studies: a cross-study evaluation of 100,000 participants. doi:10.1038/s41746-020-0224-8 Pools retention data from eight remote digital health studies run between 2014 and 2019, with more than 100,000 participants. Median retention ranged from 2 to 26 days; clinician referral added 40 days, compensation 22, having the clinical condition 7 and older age 4.
Reips, U. D., Overlander, A. T. and Bannert, M. (2025). Dropout analysis: a method for data from Internet-based research and dropR. doi:10.3758/s13428-025-02730-2 Develops a method for analysing dropout in internet-based research and releases dropR, an R package and web application. It plots dropout curves and reports chi-square values and odds ratios for points of difference, initial drop and the per cent remaining in stable states, with Kaplan-Meier survival statistics.
Donkin, L., Christensen, H., Naismith, S. L., Neal, B., Hickie, I. B. and Glozier, N. (2011). A systematic review of the impact of adherence on the effectiveness of e-therapies. doi:10.2196/jmir.1772 Reviews e-therapy studies that reported adherence. Of 69 studies, only 33 examined the relationship between adherence and outcome, and the measure related to outcome differed by target, logins in physical health and module completion in psychological health.

Check your understanding & apply it · 4 questions, 1 essay

1A deployment reports that 96 of 300 residents were still using its system in month nine, and concludes that the system 'engaged 47 per cent of residents'. Which diagnosis fits this unit?

2A team plans to estimate real-world retention by pooling the drop-out rates from published randomised trials of similar apps. On this unit's evidence, what is the likeliest error in the estimate?

3Two positions reach a review. One locates attrition in the person, citing gender and treatment expectancy. The other locates it in the study, citing the gap between reported and real-world usage. Which does this unit support?

4A team wants to test the hypothesis that a new referral pathway raises retention rather than a new app feature. Which study fits this unit's terms for settling it?

5Essay A social care partnership installs a tablet-based monitoring and reminder system in twelve residential homes. Of 300 residents enrolled, 284 had a profile created, 142 used the system at least once a week in the first month, and by the ninth month 96 residents were still using it. The nine-month report states that the system 'engaged 47 per cent of residents' and recommends purchasing 200 more tablets to 'reach the residents who chose not to participate'. Advise the partnership using this unit's argument. Say what the reported figures can and cannot support, say whether the attrition described is best explained as a property of the residents or of the deployment, set out the contested question as it bears on the purchase, state what evidence would settle it, and say what should be measured and reported over the following year. Name your sources at each step.

The figures describe use, not benefit, and they describe a falling curve. Ninety-six of 300 residents were still using the system at nine months, and the report converts that into an engagement figure without stating what use was for, which is the move Eysenbach (2005) rejects: he argues that attrition measures should be reported alongside efficacy and that the characteristics of the subpopulation for which a system works should be described, not smoothed into a single percentage. The 47 per cent also conceals two different events, the 16 residents who never had a profile created and the 188 who stopped at some point, and Baumel and colleagues (2019) show why that distinction matters, since the same programmes deliver reported usage rates a median of 4.06 times the real-world rate when users are proactively recruited and assessed. On whether this is a property of the residents or of the deployment, the unit's evidence points to the deployment. Pratap and colleagues (2020) found median retention across eight remote studies ranged from 2 to 26 days, with clinician referral worth 40 additional days, compensation 22, having the clinical condition 7 and older age 4, and Geraghty and colleagues (2013) removed most attrition simply by adding live contact, cutting it from 48.1 to 10.8 per cent in an otherwise identical cohort. Beatty and Binnion (2016) found that most participant-level variables predicted nothing, and Linardon and Fuller-Tyszkiewicz (2020) state that no baseline characteristic reliably predicted attrition. So buying tablets to reach residents who 'chose not to participate' treats a design and support problem as a resident characteristic, and Kelders and colleagues (2012) add that being studied matters too, since randomised rather than observational design predicted better adherence. The contested question the purchase turns on is therefore whether attrition belongs to the person or to the arrangement, and what would settle it is the test Baumel and colleagues (2019) ran, comparing reported usage with an objective measure for the same system, scaled up: one system, one definition of use, one follow-up window, measured in homes that receive live support and homes that do not. Over the following year the partnership should report who left and when, the retention curve rather than a single figure, the reason each person stopped, taken from the residents themselves, and any clinical or care outcome measured separately from use, using tooling such as dropR to plot and test the curve (Reips et al., 2025). If it cannot say what amount of use was supposed to produce benefit, it should test that dose rather than assume that more tablets will produce it.