Study notes · CADIO / NYU School of Global Public Health
Module 1: “MOST is a Different Way of Thinking” Authors: Linda M. Collins and Kate Guastaferro (School of Global Public Health, New York University) Format: 9 lessons (introduction, set reading, Lessons 1 to 7) · 8 videos (70 min) · 7 slide decks (PDF) · 7 knowledge checks Set reading: Collins, L. M. (2018). Optimization of behavioral, biobehavioral, and biomedical interventions: The multiphase optimization strategy (MOST). New York: Springer. (Preface and Chapters 1 and 2) Access: free via CADIO share links, or Coursera
Source inconsistency: the course's question-level feedback for two Lesson 5/6 items is the gift-card text copied from the Lesson 1 knowledge check (“This is incorrect. A gift card provided to incentivize completion of research surveys …”), which discusses an unrelated topic. Correctness marks and answer-level feedback follow the course answer key; the copied feedback is retained below, labelled as a source anomaly and not endorsed. Caption transcription artefacts in the video panel (for example “biiobehavioral”, “desiterata”, “andor”) are preserved inside narration quotes and noted where they occur.
The module's argument in one paragraph.
Module 1 sets up the case that the dominant paradigm in intervention science, the classical treatment package approach, leaves the field blind inside the black box: bundle all candidate components, set each to its higher level, evaluate the whole package in an evaluation RCT (ERCT), and hope. A well-conducted ERCT tells you whether the package beats a control, but not which components earn their keep, which interact, which are inert or counterproductive, or how to make the package more affordable, scalable and efficient. The Multiphase Optimization Strategy (MOST) imports an engineering mindset: start with a clear goal and explicit constraints, run efficient experiments to estimate the individual and combined contributions of candidate components, select components and levels to optimize within those constraints (balancing effectiveness against affordability, scalability and efficiency, intervention EASE), and only then confirm the optimized package in an ERCT. MOST proceeds through three phases, preparation, optimization, evaluationunder two principles, resource management and continual optimization. It is, in short, a different way of thinking: engineering-inspired, iterative, and committed to looking inside the black box.
Course introduction (video lesson “Introduction to the Course”): the course targets members of intervention science teams who are primarily responsible for research methods (research design and statistical analysis), often with degrees in biostatistics, statistics or quantitative psychology; the material requires graduate-level training in multivariate statistics. Six overall learning objectives run across the modules: relate MOST to the objectives of intervention research; compare and contrast the classical approach and MOST (Module 1); understand the logic of the factorial experiment (Modules 2 to 3); design highly efficient optimization trials (Module 4); recognize the wide variety of optimization trial designs (Module 5); and design and conduct optimization trials rigorously and responsibly (Module 6). The course is cumulative and should be taken in order. The reading lesson assigns the Preface and Chapters 1 and 2 of Collins (2018).
Lesson 1: Definitions of Key Concepts, Part I
The lesson defines the three building blocks used everywhere else in the course: MOST, intervention, and intervention component.
MOST (the multiphase optimization strategy): “It is an engineering inspired framework for optimizing multicomponent behavioral, biiobehavioral, biomedical and social structural interventions.” The slide deck spells the second term “biobehavioral”; the caption transcription renders it “biiobehavioral”, preserved here because the quote follows the narration caption verbatim.Narration, Lesson 1; slide 1.4
Intervention: “An intervention is a program ultimately intended to affect human health, well-being, or educational attainment by changing individual behavior or cognition or characteristics of the individual's environment.”Narration, Lesson 1; slide 1.5
An intervention may be …: “An intervention may be behavioral, biobehavioral, biomedical, or social structural.” A behavioral intervention changes cognitions or behavior; a biobehavioral intervention adds medical or biological components such as a prescription drug; a biomedical intervention is primarily medical or biological, often with behavioral aspects; a social structural intervention changes behavior by changing the environment through the legal system or the built environment. These are not discrete categories, and the point is not to label interventions but to see the range of strategies. An intervention may target any level or several: individual, family, school, neighbourhood, clinic, and beyond.Narration, Lesson 1
Examples of interventions: “Treatment of depression is a good example of a biobehavioral intervention” (talk therapy and behavioural skills plus an antidepressant). A school-based drug abuse prevention program is a multi-level behavioural intervention aimed at both the individual and school levels. A smartphone app for adolescents with Type 1 diabetes on an insulin pump helps them use the pump and maintain nutrition and exercise habits. Installing sidewalks and lighting and upgrading public parks to encourage physical activity is a social structural intervention.Narration, Lesson 1
Intervention component: “We will define an intervention component as any aspect of an intervention that can be separated out for study.” The practical test: “if it can be separated out for study and it would be useful to do so, then it can be considered a component.”Narration, Lesson 1; slides 1.12 to 1.13
Some types of intervention components: parts of the content of the intervention (“a part of the content of the intervention”, for example curriculum topics or weight-loss strategies); features that promote compliance, adherence or engagement (“a feature that promotes compliance, adherence, or engagement with the intervention”, for example reminder calls or text messages and gamification); and features aimed at improving fidelity of delivery (“features aimed at improving the fidelity of intervention delivery”, for example enhanced training or a chat window answering delivery personnel's questions). The course does not require labelling components; the list just conveys the possibilities.Narration, Lesson 1; slides 1.13 to 1.15
An important distinction: “A component is a part of the intervention included for treatment purposes. It is not included solely for research purposes.” “To be considered a component, a feature or element of an intervention must be eligible to be included when the intervention goes to scale.” Eligible is the operative word: experiments later in the course decide which components have a large enough effect to justify inclusion, so a component may be eligible yet not make the cut.Narration, Lesson 1; slide 1.16
The smartwatch scenario (intervention component or not?): in a study to optimize a weight loss intervention, participants wear a smartwatch that counts steps per day. “Can counting daily steps via a smartwatch be considered an intervention component?” If the step count is being considered for inclusion when the intervention goes to scale, “If so, then it can be considered a component.” If the smartwatch is included only to measure an outcome variable and will definitely be omitted at scale, “If so, then it cannot be considered an intervention component.” A component and an outcome measure are not mutually exclusive: if step counting is part of the intervention under study, the step count can still be used as an outcome variable.Narration, Lesson 1; slides 1.17 to 1.20
Knowledge check
MULTIPLE RESPONSE: Which of the following could be considered an example of an intervention component? Select all that apply.
☑ A prescription for antidepressant medication.
☑ A bus pass for participants to provide transportation to intervention sessions.
☑ A web site with FAQ for program staff.
☐ A gift card provided as incentive for completion of research surveys.
Feedback when incorrect: This is incorrect. A gift card provided to incentivize completion of research surveys is included for research purposes only, and will not be a part of the intervention when it goes to scale. Therefore, it cannot be considered an intervention component.
Lesson 2: Definitions of Key Concepts, Part II
The lesson introduces the running scenario and defines the RCT, the ERCT, and the classical treatment package approach.
Scenario 1: “The goal here is to identify a set of components and component levels for inclusion in this intervention.” The intervention aims to reduce viral load among HIV+ individuals who drink heavily. Candidate components: motivational interviewing (no/yes), peer mentoring (no/yes), text message support (no/yes), mindfulness meditation (no/yes), and behavioral skills training (low intensity/high intensity). “No or yes” are treated as two possible levels of a component, a concept revisited later.Narration, Lesson 2; slides 1.4 to 1.5
The usual way of building an intervention out of these components: “many scientists would immediately construct a new intervention, possibly after pilot testing the components by setting each component to the higher of the two levels and then combining the components to form a treatment package made up of all the components.” The treatment package is motivational interviewing plus peer mentoring plus text message support plus mindfulness meditation plus the high-intensity level of behavioral skills training (the caption transcription runs the compound as “highintensity”).Narration, Lesson 2; slides 1.6 to 1.8
Evaluation RCT (ERCT): the package is compared with a suitable control or comparison group in an experiment “often referred to as an RCT”; from here on “we will refer to this type of experiment as an evaluation RCT or ERCT”, reserving the label so optimization trials can be distinguished later.Narration, Lesson 2
Classical treatment package approach: “We're going to call this general approach in which a set of intervention components is identified immediately combined into a treatment package and then evaluated as a package by means of an ERCT. The classical treatment package approach.” Secondary analyses are possible afterwards, “say to try to get a sense of which components are working.”Narration, Lesson 2; slide 1.12
RCT (the acronym): “The acronym RCT can stand for randomized control trial, randomized controlled trial, that is with the ED at the end of control or randomized clinical trial.” The term typically refers to an experiment with human participants, and can appropriately refer to any design involving random assignment plus experimental control. In intervention science and medical research it has come to refer to a particular family: “In this family of experimental designs, the objective is direct comparison of the means of a limited number of experimental conditions.” Because these designs are used in intervention evaluation, the course calls them evaluation RCTs (ERCTs). Optimization trial designs, contrasted later, are introduced here as “the optimization trial or OCT” in the narration, while the slide wording is 'ORCTs'.Narration, Lesson 2; slides 1.9 to 1.11
The classical pipeline (figure): “There's an arrow running from the five components to a black cube labeled intervention. This means that the five components are combined to form an intervention package.” “There's another arrow running from the black cube labeled intervention to a rectangle labeled evaluation via ERCT”, the next step is to evaluate the package in an ERCT.Narration, Lesson 2; slide 1.12
Figure 1: The classical pipeline (Scenario 1)
Redrawn from the Lesson 2 and Lesson 3 slide decks: the five components are bundled into a single “Intervention”, which is then evaluated as a package via an ERCT.
Lesson 3: What Knowledge Can and Cannot Be Obtained from an ERCT
The lesson contrasts the classical treatment package approach (“business as usual”) with how an engineer would go about developing an intervention, using Scenario 1.
What a well-conducted ERCT can tell you: “You can learn whether a treatment package is clinically and statistically better than a suitable control treatment.” This is an important question, but not the only one that might motivate the research. The course is explicit that nothing is wrong with evaluating an intervention as a package via an ERCT: “Absolutely nothing. There is absolutely nothing wrong with evaluating an intervention as a package using an ERCT.”Narration, Lesson 3; slide 1.7
What a significant ERCT will not reveal (components): “an ERCT that finds a significant effect, will not reveal which components are making positive contributions to any overall effect that was observed.” The entire observed effect could be driven by one or two components, while others do nothing or even have an iatrogenic effect and should be eliminated.Narration, Lesson 3; slide 1.8
What a significant ERCT will not reveal (interactions): whether the inclusion of one component has an impact on the effect of another: “Maybe component A works well on average but when component B is included component A works even better.”Narration, Lesson 3; slide 1.8
What a significant ERCT will not reveal (cost): “An ERCT that finds a significant effect will not reveal whether a component's contribution justifies its cost.” Some components are more expensive than others: motivational interviewing is relatively expensive, time-consuming to implement and must be delivered by a highly trained professional, so one might want to establish that its effect is substantial enough to justify that cost.Narration, Lesson 3; slide 1.9
What a significant ERCT will not reveal (improvement): “an ERCT that finds a significant effect contains little information about how to make the intervention better, more effective, more affordable, more readily scalable, and more efficient.”Narration, Lesson 3; slide 1.9
What a non-significant ERCT will not reveal: “An ERCT that shows a non-significant effect will not help the investigator figure out whether any components are worth retaining. It could be that there were one or two good components that should be salvaged.” The ERCT also does not reveal “whether one component had a negative effect that offset the positive effect of others, producing a net null result”, and “the ERCT does not help the investigator determine specifically what went wrong and how to do things better the next time.”Narration, Lesson 3; slide 1.10
Knowledge check
MULTIPLE RESPONSE: Which of the following is something a well-conducted ERCT cannot tell us? Select all that apply.
☑ Whether the inclusion of one component has an impact on the effect of another.
☑ Whether there is a positive or negative interaction between components.
☐ Whether an intervention performs better than a control.
☑ Whether all components are needed to produce an overall treatment effect.
☑ Whether any individual component had a negative effect.
Feedback when incorrect: This is incorrect. The purpose of a well-conducted ERCT is to determine whether an intervention performs better than a control.
Lesson 4: Comparison of the Classical Treatment Package and Engineering Approaches
The lesson contrasts the two development approaches through Scenario 2 (industrial engineering) and returns to Scenario 1 to imagine behavioural scientists thinking like engineers.
Scenario 2 (leaf springs): “Here we are developing a way to manufacture truck leaf springs.” “The goal here is to choose from a set of components and component levels to optimize the amount of variability in the length of the leaf springs.” Less variability is better. The hypothetical example is based on a published article by Pignatiello and Ramberg described in a book by Wu and Hamada (the caption transcription renders the first name 'Pynatiello'; the slide citation reads 'Pignatiello'), with liberties taken.Narration, Lesson 4; slide 1.7
Scenario 2 components (levels): “Furnace temperature could be lower or higher.” “Heating time could be shorter or longer.” “Transfer time on a conveyor belt could be shorter or longer.” “Hold down time in the high pressure could be shorter or longer.” “quench oil temperature range. The oil temperature could range over lower temperatures or higher temperatures.” For each component the higher level is hypothesized to be better; the exact values do not matter in the example.Narration, Lesson 4; slide 1.8
An engineer using the classical approach: “The engineer would construct a new manufacturing process by assembling the higher levels of all the components”, then compare the process as a package with the old way to see whether it is demonstrably better, possibly followed by post-hoc analyses. The narration's “demonstrabably” and “post hoke” are caption transcription artefacts.Narration, Lesson 4; slides 1.9 to 1.10
Why engineers usually do not use the classical approach: “If the new process is better, the engineer cannot determine which components make a difference and which do not.” “If the new process is not better, the engineer cannot determine which, if any, of the components did affect an improvement and should remain under consideration.” Repeating the approach gives “no guarantee of systematic incremental improvement. So, it is not a good long run strategy.” Finally, “this approach does not take cost or other constraints into account.”Narration, Lesson 4; slides 1.11 to 1.12
The conversation about constraints: the engineer reports, “I've developed an excellent manufacturing process for leaf springs. The springs coming off the assembly line are nearly identical. You need a microscope to see the differences in length.” The plant owners ask, “That is great. What is the per spring cost?” and on hearing “$100,000 per spring” they reply, “These springs retail for about $50 a piece. We can't afford to pay more than $40 per spring for manufacturing. We won't be using your manufacturing process and you are fired.”Narration, Lesson 4; slides 1.13 to 1.14
Constraints, and why the 'best' is not always preferable: “there is a constraint on manufacturing cost. it cannot exceed $40.” The plant owners and the engineer should have established this much earlier. “So the engineers job is to develop a manufacturing process that produces the least variability in length without exceeding a budget of $40.” Is this the best possible process? “No. Usually it won't be, but it is the best one the plant can actually use.”Narration, Lesson 4; slides 1.15 to 1.16
What an engineer would do: start with a clear idea of the goal including any constraints, use the available resources to design and conduct an efficient experiment to gather the needed information, then “based on the results of that experiment, the engineer would select components and the levels of those components to balance leaf spring variability and cost to achieve the stated goal.” “This is optimization. the strategic balancing of the desiraability of the outcome against the costs that must be incurred to obtain that outcome where cost is very broadly defined.” ('Desiraability' is a caption transcription artefact.)Narration, Lesson 4; slides 1.17 to 1.18
Optimized is not the same as best possible: “optimized is not the same as best possible.” Colloquially the terms are used interchangeably, but technically they are not synonymous: “in this example, optimized is the best we can afford.” Only after optimizing would the engineer directly compare the new optimized process with the old process.Narration, Lesson 4
Optimization as a broad concept: “Optimization is a broad concept. It appropriately is defined a bit differently in different fields and different contexts. But when this term is applied appropriately, it usually involves balancing competing considerations.” Pluses, for example intervention effectiveness, against minuses such as expense, staff time and participant burden. “I strongly suggest explicitly defining the term optimization wherever you use it in your work.”Narration, Lesson 4; slides 1.20 to 1.21
Optimization of a multicomponent intervention (the definition used in MOST): “Optimization of a multicomponent intervention is the process of identifying an intervention that provides the best expected outcome obtainable within key constraints imposed by the need for affordability, scalability, andor efficiency.” ('Andor' is the caption rendering of 'and/or'.) The course notes: “This is slightly different from the definition in the book. We've recently started using the term affordability instead of economy.”Narration, Lesson 4; slide 1.22
Scenario 1 with engineers' thinking: “maybe there's an upper limit on implementation cost of $300 per participant.” The goal might be to “identify the most effective intervention within an implementation cost of less than $300 per participant.” Each component is a candidate for inclusion: “it is not a foregone conclusion that any particular component will be included in the intervention at any particular level. They are all candidates but might not all be selected.” The scientist designs an efficient experiment to gather the needed information, usually the individual and combined effects of the candidate components, then selects components or levels to balance effectiveness and cost, and only then evaluates the optimized intervention in an ERCT, and only if indicated (an ERCT would probably waste resources if not many candidate components proved effective).Narration, Lesson 4; slides 1.23 to 1.25
Why 'selecting components / component levels': “actually it's always a matter of selecting component levels.” If the levels are no and yes and no is selected, the component is not included; if the levels are low intensity and high intensity, the component is included at one of those levels. “We refer to selecting components and component levels because most people find the idea of selecting components more intuitive than the idea of selecting component levels.”Narration, Lesson 4; slides 1.26 to 1.28
MOST as a framework: “Most is a framework for enabling behavioral scientists to think like engineers and optimize their interventions.” “Most integrates ideas from a number of fields. behavioral science, engineering, multivaried statistics, decision science, and health economics.” “Most is a work in progress. It's an active area of methodological research.”Narration, Lesson 4; slide 1.29
Knowledge check
MULTIPLE RESPONSE: Which of the following statements are consistent with the definition of intervention optimization covered in this lesson? Select all that apply.
☐ Optimization is the process of identifying the intervention that provides the best possible outcome that can be obtained.
☑ Optimization involves consideration of constraints.
☑ Optimization is an ongoing process.
☑ Optimization involves striving for an efficient intervention.
Feedback when incorrect: This is incorrect. Optimization is not about identifying the intervention that provides the best possible outcome; rather, it is about balancing effectiveness against other considerations.
Lesson 5: The Concept of Optimization
The lesson reviews the MOST definition of optimization and introduces intervention EASE, then contrasts optimization with evaluation.
Optimization of a multicomponent intervention (review): “Optimization of a multicomponent intervention is the process of identifying an intervention that provides the best expected outcome obtainable within key constraints imposed by the need for affordability, scalability, andor efficiency.”Narration, Lesson 5; slide 1.4
Optimization is an ongoing process: “optimization is an ongoing process rather than an end point.” The definition says the process of identifying an intervention, not the intervention: an investigator who has identified the best expected outcome obtainable for $300 per participant can continue by examining new components to arrive at a more effective intervention still deliverable for less than $300 per person.Narration, Lesson 5
Key constraints: optimization can become complicated with too many constraints, “So, it's a good idea to identify the key constraints, that is the most important ones.”Narration, Lesson 5
The fundamental tension: “It is not possible to obtain outcomes as good as would be possible if there were no constraints.” Constraints always operate; a healthcare organization that caps delivery cost at $300 per participant will not get as potent an intervention as one willing to spend any amount. “Constraints require sacrifices such as not including all the components under consideration. Optimization is about making these sacrifices strategically.”Narration, Lesson 5
Four desiderata for multicomponent interventions: effectiveness is “the extent to which the intervention does more good than harm under real world conditions”; affordability is “the extent to which the intervention is effective without exceeding budgetary constraints”, which “has been called economy in some publications”; scalability is “the extent to which the intervention can be implemented widely with fidelity” (with fidelity meaning implemented exactly as evaluated, with no ad hoc modifications); efficiency is “the extent to which the intervention avoids wasting time, money, or other valuable resources.” “An efficient intervention has no inactive components. That is, every component is having the desired effect on the outcome.”Narration, Lesson 5; slides 1.6 to 1.9
Intervention EASE: the acronym “intervention ease where the letters stand for effectiveness, affordability, scalability and efficiency.” The caption transcription renders 'desiderata' as 'desiterata' in the slide phrase “Ease is a strategic balance of the four desiterata for multicomponent interventions.” “Intervention ease is achieved by balancing effectiveness on the one hand against affordability, scalability and efficiency on the other.”Narration, Lesson 5; slide 1.10
Optimization versus evaluation: “Optimization requires examination of the performance of individual components. For this, experimental designs other than the ERCT are needed.” “By contrast evaluation requires comparison of an intervention package to a control or comparison condition. The ERCT is the way to do this.” The factorial experiment, treated at length later in the course, is one design useful in optimization.Narration, Lesson 5; slide 1.11
Knowledge check
MULTIPLE CHOICE: Which of the following statements is correct?
☐ Intervention EASE is the process of maximizing effectiveness, affordability, scalability, and efficiency. (This is incorrect. EASE does not involve maximizing all four of these attributes of an intervention simultaneously.)
☐ Intervention EASE is a strategic balance of efficiency against affordability, scalability, and effectiveness. (This is incorrect. EASE does not involve balancing efficiency against affordability and scalability.)
☐ Intervention EASE is the process of achieving the greatest degree of effectiveness attainable. (This is incorrect. EASE does not involve achieving the greatest degree of effectiveness attainable in an absolute sense.)
☑ Intervention EASE is a strategic balance of effectiveness against affordability, scalability, and efficiency.
Feedback when incorrect (source text; source anomaly, not endorsed): This is incorrect. A gift card provided to incentivize completion of research surveys is included for research purposes only, and will not be a part of the intervention when it goes to scale. Therefore, it cannot be considered an intervention component.
Lesson 6: The Three Phases of MOST
The lesson defines the three phases and the two fundamental principles of MOST.
The purpose of MOST: “to provide a comprehensive strategy for intervention optimization and evaluation.” “Most is an engineering inspired framework in which an intervention scientist estimates the individual and combined contributions of intervention components, decides which components to retain at what levels or settings to achieve intervention ease and then possibly evaluates the optimized intervention in an ERCT.”Narration, Lesson 6; slides 1.4 to 1.6
MOST is, and is not: it is “a framework for thinking through how to optimize a behavioral intervention and achieve ease” and “a practical way of approaching the engineering of a behavioral intervention so that it meets a specific optimization objective”. It is designed to make the best use of available resources, and it is new: “very new and still an open research area. Not everything has been figured out.” MOST “is not an offthe-shelf procedure that is identical for every application” (caption rendering of 'off-the-shelf'), and “Most is also not any particular experimental design”: many design types can be used, and the ERCT is one of them.Narration, Lesson 6; slides 1.6 to 1.8
Preparation phase: “the investigator lays the groundwork for optimization.” Activities: devise a detailed conceptual model or revise an existing one, “The conceptual model is similar to a logic model but even more detailed”; “The conceptual model is a model of the process to be intervened upon showing explicitly how each candidate component is hypothesized to impact this process.” Pilot testing of components or the optimization trial is conducted here. “the optimization objective is identified”: “The optimization objective is the goal of the optimization”, for example “to achieve the best expected outcome attainable for an implementation cost of $300 or less.”Narration, Lesson 6; slide 1.11
Optimization phase: “the investigator builds the optimized intervention.” The investigator conducts one or more optimization trials: “An optimization trial is an experiment conducted to gather some of the information needed for optimization.” Once the data are obtained, the investigator uses them, possibly with cost data, to identify the intervention that best meets the selected optimization objective, the optimized intervention.Narration, Lesson 6; slide 1.11
Evaluation phase: “the investigator confirms the effectiveness of the optimized intervention by means of an ERCT.” This is “usually a pretty standard two-arm ERCT” in which participants are randomly assigned to the intervention or a control/comparison condition (a wait-list control, no treatment, current standard of care, or another appropriate comparator).Narration, Lesson 6; slide 1.11
Optimization trial designs: “The factorial experiment is the most common optimization trial design.” “The fractional factorial experiment is a variant of the factorial experiment” with fewer experimental conditions but exactly the same overall sample size. SMART (sequential multiple assignment randomized trial), micro-randomized trials and system identification are useful for adaptive interventions that can be changed strategically in response to the individual or environment; fractional factorials, SMARTs and micro-randomized trials are all in the general family of factorial experiments, while system identification comes from control engineering. “other” is listed at the end because any experimental design is a possibility as long as it is selected based on the resource management principle.Narration, Lesson 6; slide 1.11
Resource management principle: “the principle that the investigator must make the best use of the resources available to conduct research.” One way is to select a highly appropriate and efficient optimization trial design: “the investigator must select the experimental design that most directly addresses the research questions at hand and makes the most efficient use of resources such as money, staff, and research participants.”Narration, Lesson 6
The decision between optimization and evaluation: the diamond between the phases poses the question “is the optimized intervention expected to be sufficiently effective?” If the answer is no, “the resource management principle suggests that it would be a better use of resources to return to the preparation phase, revise the conceptual model and identify some new components.” “Note that any components that were effective would not be discarded. So this is not starting over from the beginning. Rather it is a process of conducting research in an iterative fashion.”Narration, Lesson 6; slide 1.11
Continual optimization principle: “most offers the opportunity to optimize and reoptimize an intervention to make it better and better.” Subsequent versions of an intervention resemble software improving with every new release. In the figure, the arrow from the evaluation box back to the preparation phase indicates that “once an intervention has been evaluated, the investigator immediately returns to the preparation phase to begin improving the intervention.”Narration, Lesson 6
Figure 2: The three phases of MOST
Redrawn from the Lesson 6 slide deck (slide 1.11): the three phases of MOST (preparation, optimization, evaluation), the decision point before evaluation, and the two principle loops: resource management (no: back to preparation) and continual optimization (evaluation → preparation).
Knowledge check
MULTIPLE CHOICE: Which one of the following correctly lists the phases of MOST?
☐ Screening, refining, and confirming. (This is incorrect. Screening, refining, and confirming were the phases that appeared in the early literature on intervention optimization. The current version of MOST has been reconceptualized to be more consistent with how intervention scientists conduct research.)
☐ Preparation, refining, and evaluation. (This is incorrect. Refining is not one of the phases of MOST (although it did appear in the early literature on intervention optimization).)
☑ Preparation, optimization, and evaluation.
☐ Preparation, pilot testing, and evaluation. (This is incorrect. Pilot testing is a part of the preparation phase of MOST, not a distinct phase. Optimization trials are not pilot studies.)
Feedback when incorrect (source text; source anomaly, not endorsed): This is incorrect. A gift card provided to incentivize completion of research surveys is included for research purposes only, and will not be a part of the intervention when it goes to scale. Therefore, it cannot be considered an intervention component.
Knowledge check
MULTIPLE RESPONSE: Which of the following statements are correct? Select all that apply.
☑ According to the resource management principle, it is necessary to select the most efficient experimental design for the optimization trial.
☐ According to the resource management principle, after an optimization trial has been conducted the next step should always be the evaluation phase of MOST.
☑ According to the continual optimization principle, an effective intervention can always be improved further.
☑ According to the continual optimization principle, improving an intervention starts with reconsideration of the conceptual model.
Feedback when incorrect: This is incorrect. According to the resource management principle, after an optimization trial has been conducted the next step may or may not be an ERCT, depending on the results of the optimization trial.
Lesson 7: Why MOST is a Different Way of Thinking
Pulls the module together as a set of differences in perspective between the classical approach and MOST. The course notes these matter when communicating with intervention scientists (for example grant proposal reviewers) unfamiliar with MOST.
Reviewing the big picture: start in the preparation phase: derive a detailed conceptual model (the process being intervened on plus how each component is hypothesized to affect it), do pilot work, identify the optimization objective. Move to the optimization phase, select an optimization trial design informed by the resource management principle, conduct the trial, and based on results and possibly cost or other constraints select components and levels. “If the identified optimized intervention appears sufficiently promising, you could decide to move to the evaluation phase and conduct an ERCT. If it does not appear sufficiently promising, you would return to the preparation phase, retaining any components that demonstrated sufficient effectiveness in the optimization trial.”Narration, Lesson 7; slide 1.4
Difference 1: overall objective: “The overall objective of the classical approach is to develop an intervention that demonstrates a statistically and clinically significant effect.” By contrast, “the overall objective of most is to develop an intervention that both achieves ease, that is a strategic balance of effectiveness, affordability, scalability, and efficiency, and demonstrates a statistically and clinically significant effect.”Narration, Lesson 7; slide 1.5
Difference 2: next steps after pilot testing: “In the classical approach, after pilot testing, the next step is for the investigator to assemble the intervention and then evaluate it as a package in an ERCT.” “After pilot testing, the investigator using most instead conducts an optimization trial and identifies the optimized intervention.”Narration, Lesson 7; slide 1.6
Difference 3: experimental designs used: “The classical approach relies primarily on the ERCT” and its variants. “For the optimization phase of most, any experimental design is a possibility. The only requirement is that the design be selected based on the resource management principle.” Module 3 shows that this very often leads to the factorial design family; for the evaluation phase, primarily the ERCT and its variants are used.Narration, Lesson 7; slide 1.7
Difference 4: examining effectiveness of individual components: in the classical approach this is accomplished primarily by post-hoc analyses on data from an ERCT; “In most this is accomplished primarily by a direct experimental manipulation of components.” (The caption transcription runs the classical description as “a post hoke analyses”.).Narration, Lesson 7; slide 1.8
Difference 5: examining interactions among components: “In the classical approach, examination of interactions among components is not possible because the classical approach relies almost exclusively on the ERCT and the ERCT does not enable examination of interactions among components.” By contrast, “in most, wherever feasible, the experimenttor selects an experimental design that enables examination of interactions among components” (“experimenttor” is a caption transcription artefact).Narration, Lesson 7; slide 1.9
Difference 6: inert or counterproductive components: “In the classical approach, inert or counterproductive components are generally tolerated as long as the overall effectiveness of the intervention package can be demonstrated.” “However, in most inert and counterproductive components are generally not tolerated.” “In fact, considerable effort is spent to identify and remove such components because they reduce the efficiency of the intervention.”Narration, Lesson 7; slide 1.10
Difference 7: scalability and sustainability: “In the classical approach, scalability and sustainability are usually dealt with after evaluation of the intervention, sometimes by means of ad hoc modifications to the intervention.” In MOST “the intervention scientist can choose to build the intervention for immediate scalability and long-term sustainability by identifying constraints on implementability such as cost” and working toward an intervention that provides an acceptable level of effectiveness within those constraints.Narration, Lesson 7; slide 1.11
MOST is a different way of thinking: “It's engineering inspired, drawing on principles of industrial and control engineering. It's iterative and discovery based, seeking to make incremental improvements over time.” “it emphasizes looking inside the so-called black box of interventions to have a better idea of how they operate.” “Most aims to establish a cumulative knowledge base about which components work and do not work to advance intervention science.” “It is focused on immediate scalability”, developing interventions that are scalable before they are evaluated in an ERCT, “and provides a framework for ongoing programmatic improvement of interventions.”Narration, Lesson 7; slides 1.12 to 1.13
Dimension
Classical approach
MOST
Overall objective
To develop an intervention that demonstrates a statistically and clinically significant effect.
To develop an intervention that both achieves EASE (strategic balance of effectiveness, affordability, scalability, efficiency) and demonstrates a statistically and clinically significant effect.
Next steps after pilot testing
Assemble the intervention and evaluate it as a package in an ERCT.
Conduct an optimization trial and identify the optimized intervention.
Experimental designs used
Primarily the ERCT and its variants.
A wide variety of designs; the optimization trial design is selected by the resource management principle (very often in the factorial family); evaluation uses primarily the ERCT and its variants.
Examining effectiveness of individual components
Primarily via post-hoc analyses on ERCT data.
Primarily via direct experimental manipulation of components.
Examining interactions among components
Not possible, the ERCT does not enable it.
Wherever feasible, an experimental design is selected that enables it.
Inert or counterproductive components
Generally tolerated, as long as overall effectiveness of the package is demonstrated.
Generally not tolerated, considerable effort is spent to identify and remove them, because they reduce efficiency.
Scalability and sustainability
Usually dealt with after evaluation, sometimes via ad hoc modifications.
The intervention is built for immediate scalability and long-term sustainability, by identifying constraints (for example cost) and working toward acceptable effectiveness within them.
Knowledge check
MULTIPLE RESPONSE: Which of the following statements correctly state differences in perspective between the classical approach and MOST? Select all that apply:
☑ A. In the classical approach inert or counterproductive components are generally tolerated, whereas these are generally not tolerated in MOST.
☐ B. The classical approach relies primarily on the ERCT, whereas MOST requires the use of a factorial experiment.
☑ C. In the classical approach, scalability is considered after an intervention has been evaluated, whereas in MOST scalability is considered from the beginning of intervention development.
☐ D. The objective of the classical approach is to develop an intervention that demonstrates a statistically and clinically significant effect, whereas the objective of MOST is to identify an optimized intervention.
☐ E. In the classical approach pilot testing is done in the usual way, whereas in MOST pilot testing is done via the optimization trial.
Feedback when incorrect: Answer B. is incorrect. MOST does not require the use of a factorial experiment; instead, any of a wide variety of experimental designs, including but not limited to the factorial experiment, are possibilities in the optimization phase. The ERCT is typically used in the evaluation phase. Answer D. is incorrect. The objective of MOST is to identify an optimized intervention that demonstrates a statistically and clinically significant effect. Answer E. is incorrect. In MOST pilot testing is done in the preparation phase, in essentially the same manner as in the classical treatment package approach. Optimization trials are fully powered and carefully controlled experiments, not pilot studies.
Synthesis (my own reading: not part of the course)
Read from the methodological perspective, Module 1 is aimed squarely at the methods team member: the person who will design the experiment, not the person who invents the content. The module's central methodological claim is that the ERCT answers exactly one question, does the package beat a suitable control, and that this is both indispensable and insufficient. Everything MOST adds is a design response to questions the ERCT cannot answer: which components contribute, how components interact, what each costs, and how the package can be made more affordable, scalable and efficient. Lesson 3 is the pivot: the ERCT is praised, not dismissed, and the argument is that different research questions require different designs, which is what the optimization phase provides.
For readers who work with the 2021 MRC/NIHR framework for developing and evaluating complex interventions, Module 1 is best read as the optimization layer the MRC framework gestures at but does not operationalise. The MRC framework's iterative phases (develop, identify, feasibility, evaluate, implement) map onto MOST's preparation (developing the conceptual model and candidate components), optimization (efficient experiments before committing to a full trial), and evaluation (the confirmatory trial). MOST's insistence on estimating individual and combined component contributions is the answer to a classic MRC-style question: a complex intervention evaluated as a package leaves you unable to say which active ingredients produced the effect, exactly the post-hoc-analysis trap Lesson 3 describes.
Two framing notes for the methodological reader. First, the course keeps the term “optimization objective” deliberately loose: it is the goal of the optimization, and the example (best expected outcome attainable for $300 or less per participant) shows it is where constraints are made explicit and operational. The choice of which constraints count as “key” is a scientific judgement the investigator must defend, and the course says keeping the number small matters. Second, the narration's OCT versus the slides' ORCT is a minor terminology wobble worth knowing about when reading the literature: later modules settle on the language of optimization trials and their designs.
One more flag to carry forward: the course repeatedly stresses that MOST “is not any particular experimental design” and that the resource management principle governs design choice. That sets up Module 3's claim that factorial-family designs are often the efficient choice: not a dogma. The course also quietly updates Collins (2018): “economy” becomes affordability, and the early literature's phase labels (screening, refining, confirming) have been reconceptualised to the current preparation/optimization/evaluation. Quote the current labels.
Glossary
MOST
The multiphase optimization strategy: an engineering-inspired framework for optimizing multicomponent behavioral, biobehavioral, biomedical and social-structural interventions.
Intervention
A program ultimately intended to affect human health, well-being or educational attainment by changing individual behavior or cognition, or characteristics of the individual's environment.
Intervention component
Any aspect of an intervention that can be separated out for study; included for treatment purposes (not solely for research), and always eligible for inclusion when the intervention goes to scale.
Candidate component
A component under consideration for inclusion; it may or may not be included after optimization.
RCT
Randomized control/controlled/clinical trial: in intervention science, a particular family of experimental designs whose objective is direct comparison of the means of a limited number of experimental conditions (typically two).
ERCT
Evaluation RCT: the course's label for that family of designs, used in intervention evaluation.
ORCT / OCT
Optimization (R)CT: the course's later label for optimization trial designs (the Lesson 2 narration says OCT; the slide says ORCT).
Classical treatment package approach
The usual paradigm: set each component to its higher level, bundle all into a treatment package, evaluate as a package in an ERCT, and rely on post-hoc analyses for component-level insight.
Constraint
A limit imposed by the need for affordability, scalability and/or efficiency (for example a cost ceiling per participant); constraints require sacrifices, and optimization makes those sacrifices strategically.
Optimization (of a multicomponent intervention)
The process of identifying an intervention that provides the best expected outcome obtainable within key constraints imposed by the need for affordability, scalability, and/or efficiency.
Optimization objective
The goal of the optimization (for example best expected outcome attainable for $300 or less per participant).
Optimization trial
An experiment conducted to gather some of the information needed for optimization.
Intervention EASE
A strategic balance of the four desiderata, Effectiveness, Affordability, Scalability, Efficiency, balancing effectiveness against the other three.
Preparation phase
Lay the groundwork: conceptual model, candidate components, pilot work, optimization objective.
Optimization phase
Build the optimized intervention: optimization trial(s), then select components/levels meeting the objective.
Evaluation phase
Confirm effectiveness of the optimized intervention via an ERCT.
Resource management principle
Make the best use of available research resources; select the most direct, efficient experimental design; if the optimized intervention is not expected to be sufficiently effective, return to preparation (retaining effective components).
Continual optimization principle
Optimize and re-optimize; after evaluation, return to preparation to improve the intervention further.
Sources
Collins, L. M. (2018). Optimization of behavioral, biobehavioral, and biomedical interventions: The multiphase optimization strategy (MOST). New York: Springer. (Set reading: Preface and Chapters 1 and 2.)
Flay, B. R. (1986). Efficacy and effectiveness trials (and other phases of research) in the development of health promotion programs. Preventive Medicine, 15, 451 to 474. (Cited for the effectiveness desideratum.)
Pignatiello, J. J., Jr., & Ramberg, J. S. (1985). Discussion of “off-line quality control, parameter design, and the Taguchi method” by Kackar, R. N. Journal of Quality Technology, 17, 198 to 206. (Cited for the leaf spring scenario; the video caption renders the first author “Pynatiello”.)
Wu, C. J., & Hamada, M. S. (2011). Experiments: Planning, analysis, and optimization. Hoboken, NJ: Wiley. (Cited for the leaf spring scenario.)
Prepared 8 Sep 2026. Notes paraphrase the narration unless marked as verbatim quotes; figures are redrawn from the course slides; quotes follow the YouTube caption transcripts. Caption transcription artefacts are preserved inside quotes and flagged. The course's seven knowledge checks are reproduced once each with the course answer key; the course presents no separate quiz distinct from these items.