Source inconsistency: course marks these options correct but attaches unrelated EASE feedback. Correctness marks follow course answer key; feedback is retained as a source anomaly.
Study notes · CADIO / NYU School of Global Public Health
Module 5: “Rigorous and Responsible Conduct of Intervention Optimization Research” Authors: Linda M. Collins and Kate Guastaferro (School of Global Public Health, New York University) Format: 8 lessons · 8 videos (55 min) · 8 slide decks (PDF) · 6 knowledge checks · 6-item quiz Set reading: Collins, L. M. (2018). Optimization of behavioral, biobehavioral, and biomedical interventions: The multiphase optimization strategy (MOST). New York: Springer. (Chapter 6) Access: free via CADIO share links, or Coursera
The module's argument in one paragraph.
Module 5 moves from the mechanics of the factorial experiment (Modules 3 and 4) to the discipline of running one rigorously and ethically. The first three lessons are about designing the experiment so that the numbers it produces are interpretable and the trial is ethical: every factor must be independently manipulable, with sensible combinations and consistent implementation, or the results are at best hard to interpret and at worst impossible to obtain (Lesson 1); every participant must receive at least the standard of care, typically by adding a constant component, whose cost is that the results generalise only to settings that include it (Lesson 2); and when the outcome of interest lies years in the future or is a rare event, the optimisation trial can legitimately use mediator measures as proxy outcomes, but only on the strength of a well-specified conceptual model (Lesson 3). The next two lessons cover the administrative ethics of the trial: informed consent in a factorial experiment, where the IRB must be educated about the design and participants must understand they may receive a combination of components (Lesson 4), and registration of the optimisation trial in a public registry such as ClinicalTrials.gov, where the vocabulary differs (arm = experimental condition) and each trial is registered separately (Lesson 5). Lesson 6 returns to a scientific-ethical principle, clinical equipoise, generalised from the two-arm RCT to trials with many experimental conditions: what matters is the empirical scientific evidence, not the investigator's beliefs or theory. The final two lessons protect the experiment from itself: contamination, the uncontrolled sharing of treatment between participants, which shrinks effect sizes and power and is fought by minimising contact, grouping, household restrictions and cluster randomisation (Lesson 7); and protocol deviations, treatment delivered other than as assigned, which are fought by training, monitoring, technology, one-component-per-staffer delivery and an aviation-style double-checking culture (Lesson 8).
Lesson 1: Ensuring That All Factors Can Be Manipulated Independently
The lesson returns to the running HIV example (reducing viral load among HIV-positive individuals who drink heavily) with four candidate components, motivational interviewing (MI), peer mentoring (PEER), text message support (TEXT) and mindfulness meditation (MIND), each at two levels, giving a 2^4 factorial experiment with 16 experimental conditions. Its theme is the assumption on which any such experiment rests: that all factors can be manipulated independently.
The manipulability assumption: “This is the assumption that all of the factors can be manipulated independently.”Narration, Lesson 1
Two implications: “This implies first that all combinations of factor levels make sense, and second, that each level of each factor is consistently implemented.” In other words, “each level of each factor is implemented in the same manner, irrespective of which other levels of other factors it is combined with.”Narration, Lesson 1
Reviewing conditions one by one: “it's a good idea to review all the experimental conditions one by one, to make sure that each combination of levels makes sense and is possible to implement.” The running example: text messages are to come from the peer mentor, so a condition with TEXT = yes and PEER = no is probably impossible to implement.Narration, Lesson 1
What happens if the assumption is violated: “Usually, you cannot conduct the experiment.” “Other times, you technically can conduct the experiment, but you might get results that are difficult to interpret.” Dropping impossible conditions is not allowed because “This will destroy the balance property and you will no longer have an efficient factorial experiment.”Narration, Lesson 1
Consistent implementation and spurious interactions: “Violation of this assumption can produce an interaction between factors where otherwise there would be no interaction, in other words, a spurious interaction.” In practice the assumption is sometimes violated to some extent, but “It's best to try to keep this to an absolute minimum.”Narration, Lesson 1
Redrawn from the Lesson 1 slide deck: the 16 experimental conditions of the 2^4 example (MI, PEER, TEXT, MIND), with outcome label for each condition.
Condition
MI
PEER
TEXT
MIND
Outcome
1
No
No
No
No
Ŷ1
2
No
No
No
Yes
Ŷ2
3
No
No
Yes
No
Ŷ3
4
No
No
Yes
Yes
Ŷ4
5
No
Yes
No
No
Ŷ5
6
No
Yes
No
Yes
Ŷ6
7
No
Yes
Yes
No
Ŷ7
8
No
Yes
Yes
Yes
Ŷ8
9
Yes
No
No
No
Ŷ9
10
Yes
No
No
Yes
Ŷ10
11
Yes
No
Yes
No
Ŷ11
12
Yes
No
Yes
Yes
Ŷ12
13
Yes
Yes
No
No
Ŷ13
14
Yes
Yes
No
Yes
Ŷ14
15
Yes
Yes
Yes
No
Ŷ15
16
Yes
Yes
Yes
Yes
Ŷ16
Knowledge check
MULTIPLE RESPONSE: Which of the following are implied by the assumption that all the factors in a factorial experiment can be manipulated independently?
☑ All of the combinations of factor levels make sense.
☐ It is possible to implement all of the combinations of factor levels within the budget allocated for the experiment.
☑ How a level of a factor is implemented is exactly the same across experimental conditions.
☐ No interactions among factors are anticipated.
Lesson 2: Ensuring That All Participants Are Provided With At Least The Standard Of Care
The lesson shows how a factorial experiment can be made ethically sound without abandoning the design: each factor already has its own control, which can be the standard of care for that factor, and in addition a constant component can be added that all participants receive. The cost of the constant component is that its effects and interactions cannot be estimated and results only generalise to settings that include it.
Participants should never be denied the standard of care: “Participants should never be denied the standard of care.”Narration, Lesson 2
Constant component: “A constant component is a component that is not experimentally manipulated. Instead, all participants are offered this component.” In the running example, standard of care is a prescription for ART plus basic HIV education, added as a column SOC set to “yes” for all participants.Narration, Lesson 2
Why a constant component cannot be evaluated: “you will not be able to determine whether the constant component has a main effect. You also will not be able to determine whether it interacts with the candidate components.”Narration, Lesson 2
Generalisability is conditional: “any results obtained cannot be generalized to a setting in which the constant component is not provided.” Example: if behavioural skills training is the constant component, it is impossible to know whether PEER's main effect depends on it.Narration, Lesson 2
When to include a constant component: “a constant component should be included in an optimization trial only when it is a given that the constant component will be in the optimized intervention.”Narration, Lesson 2
Redrawn from the Lesson 2 slide deck: the 2^4 design with a constant component SOC (standard of care) set to Yes for all participants.
Condition
SOC
MI
PEER
TEXT
MIND
Outcome
1
Yes
No
No
No
No
Ŷ1
2
Yes
No
No
No
Yes
Ŷ2
3
Yes
No
No
Yes
No
Ŷ3
4
Yes
No
No
Yes
Yes
Ŷ4
5
Yes
No
Yes
No
No
Ŷ5
6
Yes
No
Yes
No
Yes
Ŷ6
7
Yes
No
Yes
Yes
No
Ŷ7
8
Yes
No
Yes
Yes
Yes
Ŷ8
9
Yes
Yes
No
No
No
Ŷ9
10
Yes
Yes
No
No
Yes
Ŷ10
11
Yes
Yes
No
Yes
No
Ŷ11
12
Yes
Yes
No
Yes
Yes
Ŷ12
13
Yes
Yes
Yes
No
No
Ŷ13
14
Yes
Yes
Yes
No
Yes
Ŷ14
15
Yes
Yes
Yes
Yes
No
Ŷ15
16
Yes
Yes
Yes
Yes
Yes
Ŷ16
Knowledge check
MULTIPLE RESPONSE: Which of the following statements about the constant component are correct? Select all that apply.
☑ A constant component is never considered a factor in a factorial optimization trial.
☐ The investigator should always check whether the constant component interacts with experimental factors.
☐ The results of a factorial optimization trial in which a constant component is included can be expected to generalize to situations in which that constant component is not included.
☑ Including a constant component is a good way to ensure that all participants are offered at least the standard of care.
Lesson 3: What To Do if the Primary Outcome is Years in the Future
When the outcome of ultimate interest will not be observable for years (drug use in 11th grade after a 5th-grade program; Alzheimer's onset before age 90) or accumulates slowly as a rare event (heart attacks over 20 years; deaths by suicide by age 30), an optimisation trial cannot wait. MOST remains usable if the conceptual model is well specified, and mediator measures can serve as proxy outcomes, with three cautions.
Outcomes years in the future: “the program is delivered in 5th grade, and the outcome is drug use behavior in 11th grade” Alzheimer's prevention is the other example: intervention in the late 30s/early 40s, outcome onset before age 90. “It would take a long time to conduct any experiment because of the need to wait for the outcome.”Narration, Lesson 3
Rare events: “Sometimes the outcome is a relatively rare event and you have to wait a long time to accumulate enough events.” Examples: cardiac events over 20 years; deaths by suicide by age 30.Narration, Lesson 3
MOST still works: “Yes, if you have a well-specified conceptual model.” In these circumstances you must rely heavily on the conceptual model, because the optimization trial cannot wait for the distal outcome.Narration, Lesson 3
Mediators as proxy outcomes: “Instead of using HIV viral load as the outcome variable for the optimization trial, you could use measures of one or more mediators.” “measures of these variables could serve as proxy outcomes” because the intervention must change them first, so they are likely to show change sooner.Narration, Lesson 3
Reason 1: test the whole conceptual model: “you gather data that can be used to test the fit of the entire conceptual model to the observed data.”Narration, Lesson 3
Reason 2: the model could be wrong: “your conceptual model could be wrong and you would not be able to tell.” “You could successfully change a mediator, but that mediator might not, in turn, have an effect on the outcome.”Narration, Lesson 3
Reason 3: mediator effects are larger: “It is highly likely the effect on mediators will be greater in magnitude than the effect on the outcome.” “if the effect on the mediators is small, there is little hope of a substantial effect on the outcome.”Narration, Lesson 3
Keep in mind: “it's always preferable to optimize based on the outcome if you can.” When you cannot, “develop the conceptual model carefully and thoughtfully and keep refining it as the opportunity arises.”Narration, Lesson 3
Knowledge check
MULTIPLE RESPONSE: In which of the following situations would it be reasonable to consider using a measure of a mediator as an outcome of an optimization trial? Select all that apply.
☐ Optimization of an intervention to produce weight loss in obese adults.
☑ Optimization of a program to prevent strokes in adults with high blood pressure.
☐ Optimization of a smoking cessation program aimed at adults.
☑ Optimization of a program aimed at first-year college students to increase the probability that they will complete their undergraduate education within six years.
Lesson 4: Informed Consent in a Factorial Optimization Trial
A factorial experiment is human subjects research and requires informed consent. The lesson situates the factorial experiment within IRB review, noting that the investigator's experience is close to a standard two-arm RCT but that the design itself may need explaining, and illustrates consent documents with two examples, including the itMatters study.
The Institutional Review Board: “the IRB is an entity tasked with protecting the rights and welfare of human research subjects.” Also known as independent ethics committees, ethical review boards or research ethics boards.Narration, Lesson 4
A factorial experiment is human subjects research: “A factorial experiment is designated as human subjects research and would therefore require informed consent.”Narration, Lesson 4
IRB experience is like an RCT: “your experience as an investigator will not be dissimilar to what you would encounter in a standard two-arm RCT.”Narration, Lesson 4
Educating the IRB: “you may need to do some additional education to your IRB to some degree.” By default, “the factorial experiment will require you to explain more experimental conditions.” For example, the IRB may require you to explain all 32 conditions of a 2^5.Narration, Lesson 4
What participants need to know: “participants need to know what they can expect if they choose to participate in your study.” For example, the consent document may explain that they may receive a combination of components.Narration, Lesson 4
itMatters consent was to the research: “Participants thus provided informed consent to participate in the research survey, not the intervention.” All first-year students had access to the intervention regardless of research participation; content outside the assigned condition was made available at the end of the study.Narration, Lesson 4
Best practice 1: a lay handout: “we have found it helpful to provide participants, and those who are responsible for recruitment, a handout to help explain the technical elements of the jargon-laden informed consent document.”Narration, Lesson 4
Best practice 2: consent is not different: “the process of obtaining consent in a factorial optimization trial is really not different from how you would do so in a standard two-arm RCT.”Narration, Lesson 4
Lesson 5: How to Enter an Optimization Trial in a Trial Registry
A factorial experiment meets the NIH definition of a clinical trial, so it must be registered in a national public registry such as ClinicalTrials.gov. The lesson works through the registration of the itMatters study, flagging differences in vocabulary (arm vs experimental condition) and the practical burden of registering each trial separately and entering results by arm.
NIH definition of a clinical trial: “A clinical trial is a research study in which one or more human subjects are prospectively assigned to one or more interventions (which may include placebo or other control) to evaluate the effects of those interventions on health-related biomedical or behavioral outcomes.”Narration, Lesson 5, quoting the NIH
A factorial experiment is a clinical trial: “a factorial experiment is considered a clinical trial.” So it must be registered in a national, publicly available registry such as ClinicalTrials.gov, written accessibly for patients, families, researchers and study record managers.Narration, Lesson 5
Arm = experimental condition: “The equivalent term in a factorial experiment is an experimental condition.”Narration, Lesson 5
Intervention = combination of factors: “In the factorial experiment this is equivalent to the combination of factors in an experimental condition.”Narration, Lesson 5
Each trial is registered separately: “each of these trials had to be registered in the ClinicalTrials.Gov registry separately, even the two optimization trials with the same design.” In itMatters: two 2^5 optimization trials (Wyrick et al. 2020) plus an RCT evaluation trial (Tanner et al. 2021).Narration, Lesson 5
Registration is tedious; results come later: “this process can be very tedious when you sit down to enter the trial, especially when you have a 2^5 factorial experiment.” At the end of the trial, participant data and results must be entered by experimental condition (arm). “Plan ample time to enter and review this.”Narration, Lesson 5
Lesson 6: Equipoise in Optimization Trials
Clinical equipoise is generalised from the two-arm RCT to an optimisation trial with many experimental conditions. It is a property of the empirical scientific evidence, not of the investigator's beliefs or of theory, and there are two concrete ways to establish it in a factorial optimisation trial.
Freedman's clinical equipoise: “if there is genuine uncertainty within the expert medical community - not necessarily on the part of the individual investigator- about the preferred treatment.” (Freedman 1987). “What matters is the body of scientific evidence that exists about whether the experimental treatment or the control treatment is preferred.”Narration, Lesson 6, quoting Freedman (1987)
Equipoise in a two-arm ERCT: “In a two-arm ERCT clinical equipoise means there is not a convincing body of scientific evidence supporting the idea that either the experimental treatment or the comparator treatment is preferred.”Narration, Lesson 6
Equipoise in an optimization trial: “Clinical equipoise exists if there is no empirical scientific consensus about which of these versions of a treatment is preferred over the others.” With a 2^4 factorial (16 experimental conditions), equipoise exists if no conclusive body of evidence shows one or more treatments preferred to the others.Narration, Lesson 6
Not about the investigator or theory: “this is not about what you as the investigator believe.” “equipoise is not about what theory predicts. Instead, equipoise is about what the empirical scientific evidence to date demonstrates.”Narration, Lesson 6
Two ways to establish equipoise: “you can review all of the effects, that is, all of the main and interaction effects, and determine that the null hypothesis is scientifically plausible in each case.” Alternatively, “you can review the entire list of experimental conditions and determine that there is no empirical scientific basis for determining that one or more of the conditions is expected to produce a better outcome than any other.”Narration, Lesson 6
Knowledge check
MULTIPLE CHOICE: True or false: Clinical equipoise is determined based on empirical scientific evidence.
☑ True Original course feedback (source anomaly, not endorsed):This is incorrect. EASE does not involve maximizing all four of these attributes of an intervention simultaneously.
☑ False Original course feedback (source anomaly, not endorsed):This is incorrect. EASE does not involve balancing efficiency against affordability and scalability.
Source anomaly note for the Lesson 6 knowledge check: the course answer key marks both “True” and “False” as correct for this True/False item, and attaches EASE feedback to each option that reads “This is incorrect.” The answer marks below preserve the course key exactly; the retained feedback is labelled as a source anomaly rather than presented as endorsed content.
Lesson 7: How to Avoid Contamination
Contamination is uncontrolled contact between participants that exposes some of them to treatment from other conditions. It reduces effect sizes and power, and the lesson gives both examples of how it arises and practical measures to prevent it, including the power trade-off of cluster randomisation.
Contamination: “Contamination occurs when uncontrolled contact between participants results in some participants being exposed to treatment that is not a part of the treatment in their randomly assigned condition, but is a part of the treatment in other randomly assigned conditions.”Narration, Lesson 7
Contamination in both trial types: “Contamination can occur both in optimization and evaluation trials.” Optimization trials are not more prone to it per se, but with more experimental conditions “maintaining good experimental control, including preventing contamination, is in general, a bit more of a challenge.”Narration, Lesson 7
How contamination happens: “In any situation like these, where participants have the opportunity to converse, they may share information about treatments they are receiving.” Examples: waiting-room chat, group-delivered components, students in the same school; participants may even share the treatment itself.Narration, Lesson 7
Effect of contamination: “contamination tends to reduce effect sizes and therefore reduce statistical power.” At the extreme, “essentially all participants receive the same treatment.”Narration, Lesson 7
Measures to avoid contamination: “Keep unstructured contact between participants to an absolute minimum so that they don't have an opportunity to compare notes about what treatments they are or are not getting.” If components are delivered in groups, “keep all group members in the same experimental condition”; “Never allow more than one individual from the same household to participate in an experiment unless, of course, household is a unit of random assignment.”Narration, Lesson 7
Cluster randomization and the power trade-off: “you may wish to weigh the likely reduction in power that would result from contamination against the loss of power that would result from between- cluster randomization.” “Contamination reduces power, but so does between- cluster randomization.”Narration, Lesson 7
Knowledge check
MULTIPLE RESPONSE: Which of the following are ways of avoiding contamination in an optimization trial? Select all that apply.
☑ Minimize unstructured contact between experimental participants. Original course feedback (source anomaly, not endorsed):This is incorrect. EASE does not involve maximizing all four of these attributes of an intervention simultaneously.
☑ Allow at most one person per household to participate in the experiment. Original course feedback (source anomaly, not endorsed):This is incorrect. EASE does not involve balancing efficiency against affordability and scalability.
☑ Use cluster randomization.
☐ Answer participant questions efficiently by holding an open forum including all participants.
Lesson 8: How to Guard Against Protocol Deviations
Protocol deviations occur when treatment is not delivered as assigned. The lesson distinguishes deviations from contamination, separates accidental from deliberate deviations, and gives a detailed menu of prevention strategies built on training, monitoring, technology and a double-checking culture.
Protocol deviation: “Protocol deviations occur when a participant who was randomly assigned to receive a treatment associated with a particular experimental condition is not provided with that treatment or, a participant who was not randomly assigned to receive a treatment is provided with that treatment, or, in general, a treatment is delivered in a manner other than the intended manner.”Narration, Lesson 8
Deviations vs contamination: “Protocol deviations occur because experimental procedures (protocol) are not followed by intervention delivery staff, or a computer program or program software that incorrectly follows experimental procedures.” By contrast, “contamination occurs when uncontrolled contact between participants leads to sharing of information and resources.” Contamination is possible even with perfect protocol adherence; deviations are not.Narration, Lesson 8
Accidental deviations: “Accidental protocol deviations occur when a mistake is made, for example, because staff are insufficiently trained or monitored, so they make errors when delivering the intervention; instructions to those delivering the intervention are confusing or incomplete, which confuses the staff; there are mix-ups about which experimental condition a participant has been assigned to, so that the staff delivering the intervention may deliver the wrong treatment; or a computer program causes the wrong treatment to be delivered or no treatment to be delivered.”Narration, Lesson 8
Deliberate deviations: “Deliberate protocol deviations occur when staff take it upon themselves to provide or withhold treatment rather than follow the prescribed experimental protocol.” Usually done with the best of intentions; the course example is staff adding mindfulness meditation for an anxious participant assigned to condition 3, which does not include it.Narration, Lesson 8
The delicate balance in training: “they have to maintain two opposing points of view, which is difficult for anybody.” Staff must believe in the candidate components to deliver them well and simultaneously accept withholding components not in the assigned condition; training covers the scientific method, random assignment and clinical equipoise.Narration, Lesson 8
Tips for avoiding deviations: “Be sure to set aside time and other necessary resources for this.” Use technology such as REDCap, monitor and review recorded sessions, meet regularly with staff, keep deviations open for discussion, assign one component per staff person where possible, automate delivery where it makes sense, and adopt the “aviation approach”: “everything important is done or checked by at least two different people independently” in much the same way the pilot and co-pilot check equipment before a flight.Narration, Lesson 8
Knowledge check
MULTIPLE RESPONSE: Which of the following are ways of avoiding protocol deviations in an optimization trial? Select all that apply.
☑ Train staff thoroughly. Original course feedback (source anomaly, not endorsed):This is incorrect. EASE does not involve maximizing all four of these attributes of an intervention simultaneously.
☐ Encourage staff to use their own judgement about what treatment is appropriate for a given participant. (This is incorrect. EASE does not involve balancing efficiency against affordability and scalability.)
☑ Use technology to deliver intervention components where applicable.
☑ Have each staff person be responsible for delivery of a single component.
Module quiz (self-test)
Reveal the 6 quiz items with answers
MULTIPLE RESPONSE: Which of the following are implied by the assumption that all the factors in a factorial experiment can be manipulated independently?
☑ All of the combinations of factor levels make sense.
☐ It is possible to implement all of the combinations of factor levels within the budget allocated for the experiment.
☑ How a level of a factor is implemented is exactly the same across experimental conditions.
☐ No interactions among factors are anticipated.
MULTIPLE RESPONSE: Which of the following statements about the constant component are correct? Select all that apply.
☑ A constant component is never considered a factor in a factorial optimization trial.
☐ The investigator should always check whether the constant component interacts with experimental factors.
☐ The results of a factorial optimization trial in which a constant component is included can be expected to generalize to situations in which that constant component is not included.
☑ Including a constant component is a good way to ensure that all participants are offered at least the standard of care.
MULTIPLE RESPONSE: In which of the following situations would it be reasonable to consider using a measure of a mediator as an outcome of an optimization trial? Select all that apply.
☐ Optimization of an intervention to produce weight loss in obese adults.
☑ Optimization of a program to prevent strokes in adults with high blood pressure.
☐ Optimization of a smoking cessation program aimed at adults.
☑ Optimization of a program aimed at first-year college students to increase the probability that they will complete their undergraduate education within six years.
MULTIPLE CHOICE: True or false: Clinical equipoise is determined based on empirical scientific evidence.
☑ True Original course feedback (source anomaly, not endorsed):This is incorrect. EASE does not involve maximizing all four of these attributes of an intervention simultaneously.
☐ False (This is incorrect. EASE does not involve balancing efficiency against affordability and scalability.)
MULTIPLE RESPONSE: Which of the following are ways of avoiding contamination in an optimization trial? Select all that apply.
☑ Minimize unstructured contact between experimental participants. Original course feedback (source anomaly, not endorsed):This is incorrect. EASE does not involve maximizing all four of these attributes of an intervention simultaneously.
☑ Allow at most one person per household to participate in the experiment. Original course feedback (source anomaly, not endorsed):This is incorrect. EASE does not involve balancing efficiency against affordability and scalability.
☑ Use cluster randomization.
☐ Answer participant questions efficiently by holding an open forum including all participants.
MULTIPLE RESPONSE: Which of the following are ways of avoiding protocol deviations in an optimization trial? Select all that apply.
☑ Train staff thoroughly. Original course feedback (source anomaly, not endorsed):This is incorrect. EASE does not involve maximizing all four of these attributes of an intervention simultaneously.
☐ Encourage staff to use their own judgement about what treatment is appropriate for a given participant. (This is incorrect. EASE does not involve balancing efficiency against affordability and scalability.)
☑ Use technology to deliver intervention components where applicable.
☑ Have each staff person be responsible for delivery of a single component.
Synthesis: what Module 5 adds to the MOST story
The agent's reading: what this module does for the MOST story
The module is the field manual for running the optimisation trial that Modules 3 and 4 designed. Module 3 argued for the factorial experiment as the efficient engine of the optimisation phase and Module 4 supplied its statistics; Module 5 supplies the operational and ethical discipline without which the statistics are meaningless. Each lesson is a failure mode of an optimisation trial and its remedy: an impossible or inconsistently implemented condition set (Lesson 1), participants denied standard of care (Lesson 2), a distal outcome the trial cannot afford to wait for (Lesson 3), an unprepared IRB and unclear consent (Lesson 4), an unregistered trial (Lesson 5), a trial run without genuine scientific uncertainty (Lesson 6), a trial leaking treatment between conditions (Lesson 7), and a trial whose delivery staff improvise (Lesson 8).
The conceptual model is the load-bearing asset when time is short. Lesson 3 is the module's most technical: when the outcome of ultimate interest is years away, the optimisation trial trades distal outcomes for proximal ones, and the validity of that trade rests entirely on the accuracy of the conceptual model linking components to mediators to outcomes. The three reasons to prefer the outcome whenever possible (test the whole model, the model may be wrong, mediator effects overstate outcome effects) are a checklist that belongs in any protocol that proposes mediator-based optimisation.
Constant components are an ethical floor with a generalisability ceiling. The standard-of-care lesson shows the cleanest way to reconcile the factorial experiment with research ethics, add the standard of care as a constant component so no participant is denied it, while being explicit about the price: no estimate of the constant component's main effect or interactions, and results conditional on its presence. The decision rule (include it only if it will be in the optimised intervention) is the same resource-management logic applied to ethics.
Equipoise is evidence, not attitude. The Lesson 6 treatment is a useful corrective for optimisation researchers, who by definition believe their candidate components work: equipoise is about the empirical scientific record, so strong prior belief and rich theory do not remove the obligation to establish that no condition is demonstrably preferred. The two operational tests (null plausible for every effect; no empirical basis to prefer any condition) turn a philosophical concept into a protocol step.
Contamination and protocol deviations are the two ways an experiment can quietly stop being an experiment. The module's distinction matters: contamination is a property of participants sharing treatment (possible even with perfect protocol adherence), while protocol deviations are a property of delivery (possible even with perfect separation of participants). The remedies differ accordingly, separation and cluster randomisation for contamination; training, monitoring, automation and independent double-checking for deviations, and both are framed as power and interpretability problems, not just compliance problems.
Glossary
Independently manipulable factors
the assumption underlying any factorial experiment: all combinations of factor levels make sense and each level of each factor is consistently implemented.
Spurious interaction
an interaction produced by violating consistent implementation, where none would otherwise exist.
Balance property
the property of a complete factorial experiment that each factor and its interactions are estimated with equal efficiency; eliminating conditions that do not make sense destroys it.
Constant component
a component that is not experimentally manipulated; all participants receive it, typically the standard of care.
Standard of care
the accepted minimum treatment for the population; participants in an optimisation trial must never be denied it.
Mediator / proxy outcome
a variable the intervention is hypothesised to change first, which can serve as an outcome when the distal outcome is years away or rare; valid only under a well-specified conceptual model.
Conceptual model
the hypothesised causal chain from intervention components through mediators to the outcome of ultimate interest.
IRB (Institutional Review Board)
the institutional body protecting the rights and welfare of human research subjects; also called independent ethics committee, ethical review board or research ethics board.
Informed consent
the process, required for human subjects research, by which participants learn what they can expect (in a factorial experiment, possibly a combination of components) and agree to participate.
Clinical trial
per the NIH definition, a research study in which one or more human subjects are prospectively assigned to one or more interventions to evaluate their effects on health-related outcomes; a factorial experiment meets this definition.
Arm / experimental condition
ClinicalTrials.gov's “arm” designates what the factorial literature calls an experimental condition; “intervention” there corresponds to the combination of factors in a condition.
Clinical equipoise
genuine uncertainty within the expert medical community about which treatment is preferred (Freedman 1987); in an optimisation trial, no empirical scientific consensus that any version of the treatment is preferred.
Contamination
uncontrolled contact between participants that exposes some to treatment from other conditions; reduces effect sizes and power.
Protocol deviation
a participant not receiving the treatment to which they were assigned, receiving a treatment to which they were not assigned, or receiving treatment in a manner other than intended.
Accidental / deliberate protocol deviation
deviations caused by mistake (insufficient training, confusing instructions, mix-ups, computer error) vs staff intentionally providing or withholding treatment, usually with good intentions.
Cluster randomisation
assigning entire clusters (e.g. therapy groups, households, schools) to experimental conditions; prevents within-cluster contamination but itself reduces power.
Aviation approach
the practice of having everything important done or checked by at least two different people independently, as pilots and co-pilots check equipment before a flight.
Sources
Lesson 1: Ensuring That All Factors Can Be Manipulated Independently (8:55): video · slide deck (PDF)
Lesson 2: Ensuring That All Participants Are Provided With At Least The Standard Of Care (5:05): video · slide deck (PDF)
Lesson 3: What To Do if the Primary Outcome is Years in the Future (9:25): video · slide deck (PDF)
Lesson 4: Informed Consent in a Factorial Optimization Trial (7:39): video · slide deck (PDF)
Lesson 5: How to Enter an Optimization Trial in a Trial Registry (7:14): video · slide deck (PDF)
Lesson 6: Equipoise in Optimization Trials (4:13): video · slide deck (PDF)
Lesson 7: How to Avoid Contamination (5:29): video · slide deck (PDF)
Lesson 8: How to Guard Against Protocol Deviations (7:16): video · slide deck (PDF)
References cited in the lessons:
Collins, L. M. (2018). Optimization of behavioral, biobehavioral, and biomedical interventions: The multiphase optimization strategy (MOST). New York: Springer. (Chapter 6 is the set reading for Module 5.)
Collins, L. M., Kugler, K. C., & Gwadz, M. V. (2016). Optimization of multicomponent behavioral and biobehavioral interventions for the prevention and treatment of HIV/AIDS. AIDS and Behavior, 20, 197 to 214.
Freedman, B. (1987). Equipoise and the ethics of clinical research. New England Journal of Medicine, 317, 141 to 145.
Gwadz, M. V., Collins, L. M., Cleland, C. M., Leonard, N. R., Wilton, L., Gandhi, M., Braithwaite, R. S., Perlman, D. C., Kutnick, A., & Ritchie, A. S. (2017). Using the multiphase optimization strategy (MOST) to optimize an HIV care continuum intervention for vulnerable populations: a study protocol. BMC Public Health, 17, 1 to 20.
Tanner, A. E., Guastaferro, K., Wyrick, D. L., Milroy, J. J., Rulison, K., Bhandari, S., Thorpe, S., Ware, S., Miller, A. M., & Collins, L. M. (2021). A hybrid evaluation-optimization trial to evaluate an intervention targeting the intersection of alcohol and sex in college students and simultaneously test an additional component aimed at preventing sexual violence. Annals of Behavioral Medicine, 55(12), 1184 to 187.
Wyrick, D. L., Tanner, A. E., Milroy, J. J., Guastaferro, K., Bhandari, S., Kugler, K. C., Thorpe, S., Ware, S., Miller, A. M., & Collins, L. M. (2020). itMatters: Optimization of an online intervention to prevent sexually transmitted infections in college students. Journal of American College Health, 1 to 11.
Study notes prepared from the course's narrated videos, slide decks, and in-course questions. Quoted definitions are verbatim from the narration unless marked “Slides.” The synthesis section is the notes author's reading, not course content. The two redrawn tables and the conceptual-model figure are reconstructions by the notes author from the course slide montages and are labelled as such.