Feasibility and the pilot: what a small study can and cannot tell you

A pilot study buys information about whether a study can be run. Its product is not a finding about the world but measured facts about procedure: how many agreed, how many stayed, whether the apparatus worked.

This sixth of ten units argues that a pilot is an instrument for establishing whether the main study can be run, so reading its underpowered comparison as a finding, or its effect size as a planning figure, is a common misuse, and the seventh unit asks what an ethics application adds to the plan. Its purpose is feasibility (Leon, Davis and Kraemer, 2011; Eldridge et al., 2016). Its product is procedural measurement with a precision attached (Teresi, Yu, Stewart and Hays, 2022). Its effect size is the least usable figure it produces (Kraemer et al., 2006). Its size follows from what it is for (Teare et al., 2014).

The pilot is an instrument, and its purpose is a question about the main study

A pilot examines the feasibility of an approach intended for a larger study, and it can evaluate recruitment, randomisation, retention, assessment, methods and implementation.

A pilot study is not a hypothesis testing study. Safety, efficacy and effectiveness are not evaluated in a pilot.

Feasibility results do not necessarily generalise beyond the pilot's inclusion and exclusion criteria, so a recruitment figure is evidence about the route trialled, not about participants in general.

Eldridge and colleagues (2016) built the terminology through a Delphi survey, an open meeting, a consensus process and 27 empirical studies. Their framework makes pilot studies a subset of feasibility studies. The terms proved impossible to make mutually exclusive: feasibility asks whether something can be done, whether to proceed and how, while a pilot conducts part of a future study on a smaller scale.

Arain, Campbell, Cooper and Lancaster (2010) screened seven journals, the registered clinical research network portfolio, funders and editors, and found 54 pilot or feasibility studies. Of those, 81 per cent incorporated hypothesis testing, and only eight of 90 earlier pilots led to a main study.

A feasibility result is an indicator with an interval

Thabane and colleagues (2010) set out what a pilot is used to assess: safety, recruitment potential, coordination across sites and the team's experience with the method. Teresi, Yu, Stewart and Hays (2022) turn that into indicators. Earlier guidance rarely said what a feasibility indicator looks like, so they name the set: recruitment, retention, procedure fidelity, and acceptability, adherence and engagement. Acceptability includes whether a participant who uses a screen reader can complete the procedure. Because each indicator is estimated from a small sample, it arrives with a confidence interval, which is what a reader should see printed beside the figure.

Blatch-Jones, Pek, Kirkpatrick and Ashton-Key (2018) audit a major funder, reviewing pilots funded by the Health Technology Assessment programme from 2010 to 2014. In 15 stand-alone studies, they assessed testing recruitment in 100 per cent and feasibility in 83 per cent.

Recruitment is where the numbers are unforgiving. Walters and colleagues (2017) identified 151 individually randomised trials from 787 reports. The final target was achieved in 56 per cent of them, and more than 80 per cent of it in 79 per cent. Median recruitment was 0.92 participants per centre per month, and median retention was 89 per cent. An unmeasured recruitment figure is an intention.

The effect size the pilot cannot supply

Kraemer and colleagues (2006) show that using pilot data for power calculations fails in both directions. The most likely outcomes are that studies worth performing are aborted, and that those not aborted are underpowered. Their caution is aimed at that one use rather than at the pilot itself, which they accept for other purposes. Leon, Davis and Kraemer (2011) give the reason. Contrary to tradition, a pilot does not provide a meaningful effect size estimate for planning later studies, because of the imprecision inherent in data from small samples. Teresi, Yu, Stewart and Hays (2022) put the same point in intervals: most pilots should not estimate effect sizes or provide power calculations. Emanuel, Wendler and Grady (2000) place scientific validity among the seven requirements that make clinical research ethical. A comparison designed so that it cannot answer its question therefore fails an ethical requirement, not only a methodological one.

How large a pilot has to be, when size follows purpose

Teare and colleagues (2014) simulate how large an external pilot must be to estimate the parameters the main trial needs. The relative gain in precision of the pooled standard deviation is under 10 per cent for each five subjects added per group, once the total reaches 70. Adjusting the main study for a small pilot's imprecision can produce an excessively large main study. They recommend at least 70 measured subjects, 35 per group, for the pooled standard deviation. They recommend 60 to 100 where an event rate must be estimated, and at least 120 where the outcome is binary. Whitehead, Julious, Cooper and Campbell (2016) choose the size so that pilot and main trial together are as small as possible. They recommend 75, 25, 15 and 10 per arm for standardised effects that are extra small, at or below 0.1, small at 0.2, medium at 0.5 and large at 0.8.

Contested question: may a pilot's numbers plan the main study, and is a pilot a distinct object at all?

The first dispute is the sharpest here: the same arithmetic supports both sides. Kraemer and colleagues (2006) put pilots out of the business of guiding power calculations, on the argument that decisions taken from an imprecise estimate destroy useful studies and protect underpowered ones. Leon, Davis and Kraemer (2011) and Teresi, Yu, Stewart and Hays (2022) hold that a pilot's effect size should not plan the main study. Whitehead, Julious, Cooper and Campbell (2016) and Teare and colleagues (2014) reach the opposite conclusion. A pilot can inform the main study's size if it is designed and sized for that purpose, and if the planning rule accounts for its imprecision. The two-stage calculation and the larger-pilot recommendation both do that.

What would settle it is a comparison of routes against a fixed target. A simulation would plan the same main study in four ways: by ignoring the pilot, by using the pilot's standard deviation raw, by inflating it, and by designing the pilot as a two-stage procedure. Effect sizes would come from human-computer interaction results, and the simulation would report achieved power and total sample size. None of this unit's sources performs that comparison, so a proposal must say which route it takes.

The second dispute is definitional. Arain, Campbell, Cooper and Lancaster (2010) found no clear distinction between the terms pilot and feasibility, and proposed clarifications. Eldridge and colleagues (2016) concluded that pilot studies are a subset of feasibility studies, having abandoned mutually exclusive definitions when their respondents and the wider literature disagreed. A study described as a pilot and one described as a feasibility study are therefore asking the same questions, so the word is a claim about design rather than a label. What would settle whether the distinction matters is evidence that the two terms predict different objectives or outcomes. The audits by Arain and colleagues (2010) and Blatch-Jones and colleagues (2018) show the terms in use but do not test that prediction.

Read by kind, the sides are unevenly evidenced, and not in the expected direction. The case against planning from a pilot rests on argument and guidance. The case for it rests on a simulation and a formal derivation, the only worked demonstrations in the dispute, while the two audits of published pilots are descriptive. While the dispute is open, a student should size the pilot for the decision it informs, declare the progression criteria before data collection, and print each feasibility figure with its interval. The strongest objection is Whitehead's: his derivation minimises pilot and main trial together, so the unit's rule would discard a usable calculation.

Where the argument is heading

The direction is from a small study with an intention to an instrument with a specification. Eldridge and colleagues (2016) published a CONSORT extension for randomised pilot and feasibility trials. The 26-item checklist adds pre-specified criteria for judging whether or how to proceed to a definitive trial, though it was written against CONSORT 2010. Hopewell and colleagues (2025) have since replaced that parent statement with a 30-item checklist, so the pilot extension is read alongside the new one. Teresi, Yu, Stewart and Hays (2022) published indicators that evaluate several dimensions of feasibility at once. The pilot section should be a specification for an instrument: what will be measured, in what units, at what precision, and what result would send the project back to the drawing board.

The sources this unit rests on

Each note states what the work did, what it found and how it bears on the argument. Every identifier was resolved on 2 November 2026.

• Arain, M. A., Campbell, M. J., Cooper, C. L. and Lancaster, G. A. (2010). What is a pilot or feasibility study? A review of current practice and editorial policy. doi:10.1186/1471-2288-10-67 Screened seven journals and surveyed registered studies, funders and editors: of 54 studies, 81 per cent incorporated hypothesis testing and only 8 of 90 earlier pilots led to a main study. Shows the misuse as published practice.
• Blatch-Jones, A., Pek, W., Kirkpatrick, E. and Ashton-Key, M. (2018). Role of feasibility and pilot studies in randomised controlled trials: a cross-sectional study. doi:10.1136/bmjopen-2018-022233 Reviewed Health Technology Assessment programme pilots from 2010 to 2014: recruitment was tested in 100 per cent of 15 stand-alone studies, and 92 of 161 applications reported an internal pilot. Shows what a funder reads as a feasibility result.
• Eldridge, S. M., Chan, C. L., Campbell, M. J., Bond, C. M., Hopewell, S., Thabane, L. and Lancaster, G. A. (2016). CONSORT 2010 statement: extension to randomised pilot and feasibility trials. doi:10.1136/bmj.i5239 Presents a reporting extension whose 26 items add pre-specified progression criteria and implications for progression. Shows the move from an informal small study to a reported instrument.
• Eldridge, S. M., Lancaster, G. A., Campbell, M. J., Thabane, L., Hopewell, S., Coleman, C. L. and Bond, C. M. (2016). Defining feasibility and pilot studies in preparation for randomised controlled trials: development of a conceptual framework. doi:10.1371/journal.pone.0150205 Built a definitional framework from a Delphi survey, a consensus meeting and 27 studies, abandoning mutually exclusive definitions. Concludes that pilot studies are a subset of feasibility studies asking what can be done, whether to proceed and how.
• Emanuel, E. J., Wendler, D. and Grady, C. (2000). What makes clinical research ethical? doi:10.1001/jama.283.20.2701 Sets out seven requirements that make clinical research ethical, the second being scientific validity, and holds that informed consent alone is neither necessary nor sufficient. Makes a study that cannot answer its question a failure of an ethical requirement.
• Kraemer, H. C., Mintz, J., Noda, A., Tinklenberg, J. and Yesavage, J. A. (2006). Caution regarding the use of pilot studies to guide power calculations for study proposals. doi:10.1001/archpsyc.63.5.484 Finds that where pilots guide power calculations, studies worth performing are most likely aborted and studies not aborted are underpowered. Endorses pilots for other purposes.
• Leon, A. C., Davis, L. L. and Kraemer, H. C. (2011). The role and interpretation of pilot studies in clinical research. doi:10.1016/j.jpsychires.2010.10.008 Sets out the roles and limits of pilot studies, which evaluate recruitment, randomisation, retention, assessment and implementation and are not hypothesis testing studies. Reports that a pilot gives no meaningful effect size for planning later studies and that feasibility results do not generalise beyond its criteria.
• Teare, M. D., Dimairo, M., Shephard, N., Hayman, A., Whitehead, A. and Walters, S. J. (2014). Sample size requirements to estimate key design parameters from external pilot randomised controlled trials: a simulation study. doi:10.1186/1745-6215-15-264 Simulated how precision changes as a pilot grows: the gain in precision of the pooled standard deviation is under 10 per cent per five subjects added once the total reaches 70. Recommends at least 70 subjects, 60 to 100 for an event rate and 120 for a binary outcome.
• Teresi, J. A., Yu, X., Stewart, A. L. and Hays, R. D. (2022). Guidelines for designing and evaluating feasibility pilot studies. doi:10.1097/mlr.0000000000001664 Sets out feasibility indicators, including recruitment, retention, fidelity, acceptability, adherence and engagement. Concludes that most pilots should not estimate effect sizes, provide power calculations or perform exploratory analyses of efficacy.
• Thabane, L., Ma, J., Chu, R., Cheng, J., Ismaila, A. and colleagues (2010). A tutorial on pilot studies: the what, why and how. doi:10.1186/1471-2288-10-1 A tutorial on the reasons for a pilot, its relation to proof-of-concept and adaptive designs, and CONSORT-style reporting. Sets out the assessments a pilot supports, including safety, recruitment potential, coordination and the team's experience with the method.
• Walters, S. J., Bonacho dos Anjos Henriques-Cadby, I., Bortolami, O., Flight, L., Hind, D. and colleagues (2017). Recruitment and retention of participants in randomised controlled trials: a review of trials funded and published by the United Kingdom Health Technology Assessment Programme. doi:10.1136/bmjopen-2016-015276 Reviewed 151 trials from 787 reports. The final target was achieved in 56 per cent and more than 80 per cent of it in 79 per cent, with median recruitment of 0.92 participants per centre per month and median retention of 89 per cent. Supplies the units for a recruitment measurement.
• Whitehead, A. L., Julious, S. A., Cooper, C. L. and Campbell, M. J. (2016). Estimating the sample size for a pilot randomised trial to minimise the overall trial sample size for the external pilot and main trial for a continuous outcome variable. doi:10.1177/0962280215588241 Derives the pilot size that minimises the pilot and main trial together for a continuous outcome. For 90 per cent power and two-sided 5 per cent significance, recommends 75, 25, 15 and 10 per arm for extra small, small, medium and large standardised effects.
• Hopewell, S., Chan, A.-W., Collins, G. S. and colleagues (2025). CONSORT 2025 statement: updated guideline for reporting randomised trials. BMJ. doi:10.1136/bmj-2024-081123 Replaces the CONSORT 2010 statement with a 30-item checklist that adds seven items, revises three and deletes one, and ends the use of the 2010 version. The pilot and feasibility extension still applies to CONSORT 2010, so the two are read together.