← MOST study notes index · CADIO / NYU School of Global Public Health
Study notes · CADIO / NYU School of Global Public Health
The lesson defines the conceptual model and its role as the engine of the intervention.
Generic outline. The template reads left to right: intervention components on the left, proximal mediators in the middle, and the outcome of interest on the right. Each component links to one mediator that is expected to impact short- and long-term outcomes.
Example: PTSD. A conceptual model for a hypothetical intervention to reduce PTSD symptom severity among children who have experienced child maltreatment: five candidate components (relaxation training, emotion regulation, cognitive processing, exposure and response prevention, acceptance) each target one mediator (stress biology impairment, affect and mood dysregulation, cognitive distortions, avoidance behavior, psychological inflexibility), and the right side describes how the mediators affect behaviors related to the outcome, reduced PTSD symptom severity (e.g. measured with the UCLA PTSD index). The causal flow runs left to right.
| Candidate component (e.g.) | Targeted mediator |
|---|---|
| Relaxation training (e.g., diaphragmatic breathing) | Stress biology impairment |
| Emotion regulation (e.g., skills training) | Affect and mood dysregulation |
| Cognitive processing (e.g., cognitive restructuring) | Cognitive distortions |
| Exposure and response prevention (e.g., imaginal exposure) | Avoidance behavior |
| Acceptance (e.g., mindfulness) | Psychological inflexibility |
Example: STI prevention among first-year college students. A more complex causal process. The figure includes a constant component - “information”which is not subjected to experimentation.
In the STI figure: the constant component “information” plus five intervention components (in teal) each target one proximal mediator (in blue); the theoretical and empirically-informed process of behavior change is described through a series of mediators and proximal outcomes, leading finally to the distal outcome, decreased sexually-transmitted infections. (Figure from Kugler et al., 2018.)
What a conceptual model is not. It is not a logic model: “A logic model does not link outcomes to activities and the principles of an intervention.” A logic model is a good starting point, but the conceptual model “distinctly details the explicit mechanisms by which the intervention components are expected to effect change.” It is also not a structural equation model: there are similarities, but “the conceptual model does not depict every possible statistical relationship among the components, mediators, and outcome.” Those relationships may be examined post hoc; the purpose of the model is to describe the engine of the intervention.
Why it matters. The conceptual model is critical to the rest of MOST: it guides selection of candidate components (preparation), informs the experimental design chosen for the optimization trial (optimization), guides continual optimization, and is critical in a compelling grant proposal, reviewers benefit from seeing how the intervention will change behavior. The work can form the basis of a literature review article, and the course strongly recommends including the conceptual model in the protocol paper describing the optimization trial (example: Gwadz et al., 2017). Teams revise the model multiple times, it is iterative, time-consuming and challenging, “a lot of work, but it is worth it, and you should absolutely never skip this step.”
MULTIPLE RESPONSE: Which of the following is true of conceptual models? Select all that apply.
Feedback when incorrect: A structural equation modeling diagram can be used to represent a conceptual model.This is incorrect. A structural equation modeling diagram should not be used to represent a conceptual model. Although it is always a good idea to include a conceptual model, if it is too difficult to create one, it is acceptable to skip this step. This is incorrect. Development of a conceptual model, or refinement of an existing conceptual model, is an essential part of the preparation phase of MOST and must never be skipped. Conceptual models are usually not publishable. This is incorrect. Conceptual models are often publishable, for example, they can be included in protocol articles.
How to develop a robust conceptual model: theory and evidence, specificity, forward engineering, and participant heterogeneity.
The role of theory and empirical findings. The conceptual model is explicitly theory-based; it may be informed by more than one theory, and different theories may apply to different aspects of the model. It is also informed by peer-reviewed empirical literature as much as possible.
What if a field has little theory or published empirical work? You still need a conceptual model, in this scenario it could be even more critical. “Your job is to develop one for your area of study.” In this situation you may wish to examine unpublished or grey literature, conference presentations, conduct secondary data analyses, or consider clinical practice and experience.
The conceptual model should express how the intervention components are expected to affect the theoretically and empirically informed causal process. Mediators matter: you must specify which component targets which mediator. One component targeting one mediator is recommended for clarity in design and analysis; targeting more than one is possible, and relationships can be explored analytically later.
Why not reverse-engineer? Do not start by brainstorming a list of components and then justify them by expanding the model. You could overlook important components; you might include components that do not target any causal factor; and the reverse-engineering approach is a poor basis for continual optimization. One exception: if you are optimizing an existing intervention whose components have already been selected but which has no conceptual model yet, you must build out the model knowing the components, still as theoretically and empirically informed as possible.
Dealing with participant heterogeneity. Several scenarios: (1) the intervention is aimed at a specific subgroup (e.g. pregnant women, men who have sex with men), specify this clearly in the model; (2) different subgroups may benefit from different approaches (low vs high health literacy; cultural subgroups); (3) different approaches may be needed at different times, different stages of development (pre-pubertal vs later adolescence), or different points in a time-varying process (e.g. a chronic relapsing disorder such as nicotine addiction).
MULTIPLE RESPONSE: Which of the following are best practices for development of a conceptual model? Select all that apply.
Feedback when incorrect: This is incorrect. Wherever possible, the best strategy is start by developing a causal model of the process to be intervened on.
How to derive the causal part of the conceptual model, and how to use it to identify the intervention components needed.
Recall. The conceptual model is a causal model of the process to be intervened upon; it depicts all that is known or hypothesized, plus a clear description of how the intervention will intervene on the process. In the generic template, the components on the left each target one mediator; it is the right side of the model, inside the purple box, that causes the outcome.
Hypothetical example. Adapted from Collins, Kugler, & Gwadz (2016): an intervention to reduce HIV viral load in people who drink heavily. People living with HIV who drink heavily are vulnerable because they tend to be less adherent to their treatment regimen, and alcohol use is associated with increased viral load.
Building the causal (right) part of the model. Peer-reviewed literature shows alcohol use is associated with HIV viral load, and with poor ART (antiretroviral therapy) adherence, which is also associated with viral load, this is the distal part of the causal process the intervention aims to change. Drawing on social cognitive theory: intentions to act lead to behavior change, intention to reduce alcohol use leads to less alcohol use; intention to maintain ART adherence increases adherence. To affect those intentions, health beliefs must be targeted; positive role models with lived experience may be important in developing intentions to act; and social support improves intentions. Empirical literature adds that mental health status affects alcohol use and ART adherence (most of the target population will have clinically significant distress, depression or anxiety, from coping with diagnosis, treatment, and stigma), and that behavioral skills, strategies to reduce or refuse alcohol, skills for taking medication, are important to achieving these goals.
From the causal model to the components. The causal part of the model points out which causal factors the components should aim at. The content and explicit strategies within a component are then determined and informed by behavior change theory. Returning to the example, five components were identified, each targeting one mediator. The course recommends the “Target” arrow between component and mediator to make the intended link explicit.
What each component contains. (1) Motivational interviewing, the participant examines patterns of alcohol use in the context of HIV infection and ART adherence, considers normative beliefs about alcohol use or adherence, and considers intentions to use less alcohol and adhere to ART; targets health beliefs. (2) Peer mentoring, a peer with lived experience (HIV+, past alcohol problems, currently managing alcohol at a non-problem level, adhering to ART) is in regular contact with the participant; targets positive role models. (3) Text message support, a strong social support system available at the participant's convenience; targets social support system. (4) Mindfulness meditation, training in meditation practices to reduce stress and improve mental health; targets mental health status. (5) Behavioral skills training, skills for reducing or refusing alcohol and for taking medication; targets behavioral skills.
Any component can be conceptualized at different levels of granularity, finer (more micro) or coarser (more macro).
Finer granularity. Split a component into more specific ones. Example: the text message support component could be separated into instrumental text message support and encouraging text message support, two components.
Coarser granularity. Combine components. Example: the peer mentoring and text message support components could be combined into one component with a more macro level of granularity.
| Granularity | Pros | Cons |
|---|---|---|
| Finer (micro) | Heightened precision in the links between behavior change techniques and mediators; more information about how the component works is available from the optimization trial (e.g. whether both instrumental and encouraging text messages affect the perceived strength of the social support system). | Generally more components, and therefore a more complex optimization trial. |
| Coarser (macro) | Generally fewer components and a less complex optimization trial; may be the best course of action in less mature fields. | Loss of precision in the link between behavior change techniques and mediators; less information obtained in the optimization trial; findings may be difficult to interpret. |
Regardless of granularity, when possible try not to have a component target more than one mediatorbut recognize that sometimes this is necessary and unavoidable.
MULTIPLE RESPONSE: Which of the following statements about component granularity are true? Select all that apply.
Feedback when incorrect: A conceptual model should never include a component that targets more than one mediator. This is incorrect. It is a good idea to avoid having a component target more than one mediator, but it is sometimes necessary.
Moderation is when the effect of one variable differs depending on the value or level of another; statistically, moderation is an interaction. Two varieties:
How moderation is drawn. In a figure, moderation is represented by an arrow terminating at another arrow, the moderator affects the component→mediator path (variety a: one component's mediator moderates another component's effect) or an observed variable moderates a component's effect (variety b).
Conceptual model vs the figure that represents it. The conceptual model expresses the engine of the intervention; the figure represents the key parts of that engine at a glance. A figure does not, and cannot, detail every aspect: it will not show every association, interaction, or relationship to be tested statistically. It should reflect anticipated interactions involving components, but otherwise be sparing with moderators, too many make the figure complicated and messy, defeating its purpose. In a document, include a narrative description that explains the entire model. The figure plus narrative can go in grant proposals, manuscripts for an optimization trial, protocol papers, or literature reviews.
MULTIPLE RESPONSE: Which of the following statements about moderation are true? Select all that apply.
Feedback when incorrect: This is incorrect. The figure representing a conceptual model should depict particularly critical instances of moderation, but it does not necessarily have to depict all instances.
The pilot study falls within the preparation phase of MOST. Different people define pilot studies differently; the course takes its view from Leon, Davis, & Kraemer (2011, Journal of Psychiatric Research).
The purpose of a pilot study is not to support classical hypothesis testing, and not to support decision-making beyond confirming study procedures or feasibility for a larger study. In other words, its purpose is to try out procedures, experimental, recruitment, retention, and to establish feasibility.
| Pilot study | Optimization trial | |
|---|---|---|
| Statistical power | Designed with no regard for statistical power | Designed to be adequately powered to estimate effect sizes |
| Experimental control | No need for experimental control | Carefully controlled |
| Primary purpose | To prepare for the larger subsequent trial that will support statistical inference and decision-making | To support statistical inference and decision-making, it is the larger subsequent trial |
MULTIPLE CHOICE: Which of the following statements about pilot studies are true, according to the definition of pilot study used in this course?
The final part of the preparation phase: express what is meant by intervention EASE in your particular study.
Review: the four desiderata of multicomponent interventions:
| Desideratum | Definition |
|---|---|
| Effectiveness | The extent to which the intervention does more good than harm under real-world conditions (Flay, 1986). |
| Affordability (also called economy) | The extent to which the intervention is effective without exceeding budgetary constraints. |
| Scalability | The extent to which the intervention can be implemented widely with fidelity. |
| Efficiency | The extent to which the intervention avoids wasting time, money, or other valuable resources. |
What informs it: resource limitations. The objective is informed by limitations on important and finite resources needed to deliver the intervention (not to conduct the optimization trial): money to deliver, provider or staff time, participant time, or any other resource or cost. Even without an upper limit you may want to make the best use of resources and avoid waste. It is usually not necessary to conduct a full cost analysis; often one question suffices, willingness to pay: what is the amount whoever is paying for implementation is willing to pay?
Examples of possible optimization objectives (each is a different balance of EASE):
| Objective (verbatim from slides) | EASE balance |
|---|---|
| “I want an intervention made up of all active components” | An efficient intervention; cost is not an explicit consideration. |
| “I want to select the components that will provide the most effective intervention that can be implemented for less than (say) $300 per person” | Effectiveness against affordability. There are many ways to define affordable, not always a per-participant cost. |
| Select the components providing the most effective intervention that can be completed in less than 1 hour of staff time | Effectiveness against scalability, at least with respect to provider time to implement. |
| Select the set of components representing the most cost-effective intervention that does not exceed an implementation cost of $300 per person | Effectiveness against both affordability and scalability. |
| Select the set of components representing the most time-effective intervention | Effectiveness against scalability and efficiency. |
Why specify the objective? In the optimization phase, after the optimization trial you select the winning components from the full set of candidates, based on the trial results and the optimization objective. Example: suppose the trial shows only components 1, 3, 4, and 5 had substantial effects. If cost is not a consideration, include all four. If willingness to pay is ≤ $300 per person and all four cost more than $300, use the trial results to select the subset that makes up the most effective intervention under $300, possibly omitting one or more effective components. Specifying the objective in the preparation phase ensures you collect the data you need during the trial (e.g. cost or time to implement each component), and lets you pilot-test cost-data procedures if necessary.
Can the objective change later? Enthusiastically yes. Results from the same optimization trial can optimize the intervention for different settings with different objectives, e.g. two clinics, one able to pay $300 per person and one $200, may end up with different component sets.
MULTIPLE CHOICE: Which one of following best defines Intervention EASE? Intervention EASE is:
MULTIPLE RESPONSE: Which of the following statements about moderation are true? Select all that apply.
Feedback when incorrect: This is incorrect. The figure representing a conceptual model should depict particularly critical instances of moderation, but it does not necessarily have to depict all instances.
MULTIPLE CHOICE: Which of the following statements about pilot studies are true, according to the definition of pilot study used in this course?
MULTIPLE CHOICE: Which one of following best defines Intervention EASE? Intervention EASE is:
MULTIPLE RESPONSE: Which of the following are best practices for development of a conceptual model? Select all that apply.
Feedback when incorrect: This is incorrect. Wherever possible, the best strategy is start by developing a causal model of the process to be intervened on.
MULTIPLE RESPONSE: Which of the following statements about component granularity are true? Select all that apply.
Feedback when incorrect: This is incorrect. It is a good idea to avoid having a component target more than one mediator, but it is sometimes necessary.
MULTIPLE RESPONSE: Which of the following is true of conceptual models? Select all that apply.
Feedback when incorrect: Although it is always a good idea to include a conceptual model, if it is too difficult to create one, it is acceptable to skip this step. This is incorrect. Development of a conceptual model, or refinement of an existing conceptual model, is an essential part of the preparation phase of MOST and must never be skipped. Conceptual models are usually not publishable. This is incorrect. Conceptual models are often publishable, for example, they can be included in protocol articles.
Module 2 is the answer to the question Module 1 leaves open: once you have decided to optimize rather than package-evaluate, what do you actually do first? The module's ordering is itself the method: causal model before components (forward engineering), components before trial design (granularity), trial design before the trial (piloting the optimization trial as designed), and the optimization objective before any of the data collection it will govern. For readers used to the 2021 MRC/NIHR complex-interventions framework, the preparation phase maps onto the "develop" and "identify" stages, with two emphases the MRC framework states more weakly: the conceptual model must be causal (not a logic model, not an SEM), and the optimization objective must be specified before the optimization trial so the trial collects the cost/time data the decision needs.
The constant component is the subtlest concept here. It is tempting to read "not subjected to experimentation" as "safe", the module warns the opposite: because the constant component is not manipulated, its effect on the outcome and its interactions with the manipulated components can never be estimated from the optimization trial. That is a design trade-off, not a free convenience. The STI example's "information" component is the canonical case: everyone gets it, so nobody learns what it contributes.
One translation note for behaviour-change readers: the mediators in the HIV example (health beliefs, positive role models, social support, mental health status, behavioral skills) map directly onto the behaviour-change technique families a TIDieR/BCT-specification habit would demand, the course is deliberately agnostic about which behaviour-change theory supplies the causal story, and uses social cognitive theory as an illustration, not a prescription. The "Target" arrow convention (component → mediator) is worth adopting in your own figures: it forces the one-component-one-mediator discipline the module recommends and makes moderation arrows legible when they terminate on those links.
A small course artifact worth knowing about: in the module quiz, two items' answer keys are scrambled relative to the same questions asked as knowledge checks, the granularity item marks "a component that targets more than one mediator should never be included" as correct (the lesson itself says the opposite: avoid it but sometimes necessary), and the conceptual-models item marks SEM diagrams and skipping the model as correct (the lesson says the reverse). The answer key shown here follows the module quiz as published; treat the knowledge-check keys as the source of truth for the concepts.