What is an Adaptive Implementation Intervention? Why do we need - PowerPoint PPT Presentation

Adaptive School-based Implementation of CBT Primary Aim: Compare the AII (w/Coaching+Facilitation) versus REP alone on number of SP-delivered CBT sessions delivered over 18mos. versus

Adaptive School-based Implementation of CBT Emprically Develop an More Optimized AII: (a) Is the effect of augmenting REP with Coaching moderated by school-aggregated school provider training or baseline perceptions of CBT? (b) Among schools that show a potential need for further support, is the effect of augmentation with Facilitation moderated by school-level CBT delivery during first 8 weeks post-randomization, number of barriers to CBT reported 8 weeks post-randomization, satisfaction with current implementation support, or school administrator support for adoption of innovation?

Adaptive School-based Implementation of CBT PI: Kilbourne, Co-I: Smith, Methodologist: Almirall

What did you learn in the last 8 minutes? You learned about Adaptive Implementation Interventions You learned about SMART designs for empirically developing an optimized Adaptive Implementation Intervention You learned about 2 example SMARTs

Thank you! Questions? dalmiral@umich.edu, http://www-personal.umich.edu/ ∼ dalmiral/ If you are an Education researcher: http://d3lab-isr.com/training D&I Adaptive Implementation Interventions Dec 2019 35 / 118

References about AIIs and Clustered SMARTs Kilbourne, Smith ... Almirall (2018). Adaptive School-based Implementation of CBT (ASIC): clustered-SMART for building an optimized adaptive implementation intervention to improve uptake of mental health interventions in schools., Implementation Science. NeCamp, Kilbourne, Almirall (2017). Cluster-level adaptive interventions and sequential, multiple assignment, randomized trials: Estimation and sample size considerations, Statistical Methods in Medical Research. Kilbourne, Almirall, et al. (2014). Adaptive Implementation of Effective Programs Trial (ADEPT): cluster randomized SMART trial comparing a standard versus enhanced implementation strategy to improve outcomes of a mood disorders program, Implementation Science. D&I Adaptive Implementation Interventions Dec 2019 36 / 118

More General References Lu, Nahum-Shani, Kasari, Lynch, Oslin, Pelham, Fabiano, Almirall. (2016). Comparing DTRs using repeated-measures outcomes: modeling considerations in SMART studies, Stats in Medicine. Almirall, DiStefano, Chang, Shire, Lu, Nahum-Shani, Kasari, C. (2016). Adaptive interventions and longitudinal outcomes in minimally verbal children with ASD: Role of speech-generating devices, JCCAP. Dziak, Yap, Almirall, McKay, Lynch, and Nahum-Shani (2019). A Data Analysis Method for Using Longitudinal Binary Outcome Data from a SMART to Compare Adaptive Interventions. MBR, 1-24. Nahum-Shani, Almirall, Yap, McKay, Lynch, Freiheit, Dziak (2019). SMART Longitudinal Analysis: A Tutorial for Using Repeated Outcome Measures from SMART Studies to Compare Adaptive Interventions. Psych Methods Seewald, Nahum-Shani, McKay, Almirall (under review). Sample size considerations for comparing DTRs in a sequentially-randomized trial with a continuous longitudinal outcome. Luers, Qian, Nahum-Shani, Kasari, Almirall (in progress). Longitudinal Mixed-effects Models to compare DTRs in Sequentially-randomized Trials D&I Adaptive Implementation Interventions Dec 2019 37 / 118

Even More General References Almirall, Kasari, McCaffrey, Nahum-Shani. (2018). Developing Optimized Adaptive Interventions in Education, Journal of Research on Educational Effectiveness. Almirall, Nahum-Shani, Wang, Kasari. (2018). Experimental Designs for Research on Adaptive Interventions: Singly- and Sequentially-Randomized Trials, Optimization of Multicomponent Behavioral Biobehavioral and Biomedical Interventions using MOST. L. Collins, K. Kugler (Editors). Almirall, Compton, Gunlicks-Stoessel, Duan, Murphy (2012). Designing a Pilot Sequential Multiple Assignment Randomized Trial for Developing an Adaptive Treatment Strategy.” Statistics in Medicine Kim, H. and Almirall (2016). A sample size calculator for SMART pilot studies, SIAM Undergraduate Research Journal, Vol. 9. ◮ Web-applet: https://methodologycenter.shinyapps.io/PilotShiny/ D&I Adaptive Implementation Interventions Dec 2019 38 / 118

END TALK D&I Adaptive Implementation Interventions Dec 2019 39 / 118

Extra Slides Starting Here... D&I Adaptive Implementation Interventions Dec 2019 40 / 118

Sample Size Formulae for Cluster-Randomized SMARTs For comparing 2 embedded AIs that begin with different treatments in a prototypical SMART 2 4 ( z 1 − α/ 2 + z 1 − β ) × (2 − r ) × (1 + ( m − 1) ρ ) × (1 − α 2 ) N ≥ m ∗ δ 2 where r = response rate after stage 1 impl strategy m = avg number of SPs at each school ρ = inter-school correlation in outcome α = corr(baseline cluster-level covariate, outcome) D&I Adaptive Implementation Interventions Dec 2019 41 / 118

Sample Size Formulae for Cluster-Randomized SMARTs For comparing 2 embedded AIs that begin with different treatments in a prototypical SMART 2 4 ( z 1 − α/ 2 + z 1 − β ) × (2 − r ) × (1 + ( m − 1) ρ ) × (1 − α 2 ) N ≥ m ∗ δ 2 where r = response rate after stage 1 impl strategy m = avg number of SPs at each school ρ = inter-school correlation in outcome α = corr(baseline cluster-level covariate, outcome) r = . 1 and . 5 , δ = . 5 , ρ = . 03 , m = 2 , α = . 05, N = 100, power=80% D&I Adaptive Implementation Interventions Dec 2019 41 / 118

Myths or Misconceptions about Adaptive Interventions and SMARTs D&I Adaptive Implementation Interventions Dec 2019 42 / 118

Myths or Misconceptions about Adaptive Interventions Tailoring variables cannot differ based on previous intervention An adaptive intervention must recommend a single intervention component at each decision point Adaptive interventions seek to replace clinical judgement Adaptive interventions are only relevant in treatment settings Adaptive interventions are non-standard because they involve randomization D&I Adaptive Implementation Interventions Dec 2019 43 / 118

Interventions for Minimally Verbal Children with Autism PIs: Kasari(UCLA), Almirall(Mich), Kaiser(Vanderbilt), Smith(Rochester), Lord(Cornell)

Myths or Misconceptions about SMART Studies All research on adaptive interventions requires a SMART SMARTs are an alternative to the RCT SMARTs require prohibitively large sample sizes All SMARTs require Multiple-Comparisons Adjustments All SMARTs must include an embedded tailoring variable All aspects of an embedded adaptive intervention must be randomized SMARTs are a form of adaptive research design SMARTs never include a control group SMARTs require a “business as usual” control group SMARTs require multiple consents SMARTs are susceptible to high levels of study drop-out D&I Adaptive Implementation Interventions Dec 2019 45 / 118

Interventions for Minimally Verbal Children with Autism PIs: Kasari(UCLA), Almirall(Mich), Kaiser(Vanderbilt), Smith(Rochester), Lord(Cornell)

Myths or Misconceptions about SMART Studies All research on adaptive interventions requires a SMART SMARTs are an alternative to the RCT SMARTs require prohibitively large sample sizes All SMARTs require Multiple-Comparisons Adjustments All SMARTs must include an embedded tailoring variable All aspects of an embedded adaptive intervention must be randomized SMARTs are a form of adaptive research design SMARTs never include a control group SMARTs require a “business as usual” control group SMARTs require multiple consents SMARTs are susceptible to high levels of study drop-out D&I Adaptive Implementation Interventions Dec 2019 47 / 118

Multi-Tiered Systems of Support PIs: Greg Roberts and Nathan Clemens (UT Austin)

Multi-Tiered Systems of Support PIs: Greg Roberts and Nathan Clemens (UT Austin) Let’s call this ”Academic Adaptive Intervention.”

Multi-Tiered Systems of Support PIs: Greg Roberts and Nathan Clemens (UT Austin) This study investigates the role of the self-regulation component (should it be provided in stage 1, in stage 2, or at all?) in the context of the Academic Adaptive Intervention.

Multi-Tiered Systems of Support PIs: Greg Roberts and Nathan Clemens (UT Austin) I call this a “Seemingly-Restricted SMART”. Here, a 2x2 SMART design.

You may be wondering about sample size/power?

Prototypical SMART

Sample Size Formulae with Repeated-measures Analyses For comparing 2 embedded AIs that begin with different treatments in a prototypical SMART; Hypothesis test is that there is no difference in means at the end of the study � 1 − ρ 2 � 2 4 ( z 1 − α/ 2 + z 1 − β ) N ≥ × (2 − r ) × δ 2 where r = response rate after stage 1 treatment ρ = within-person correlation in outcome D&I Adaptive Implementation Interventions Dec 2019 54 / 118

Sample Size Formulae with Repeated-measures Analyses For comparing 2 embedded AIs that begin with different treatments in a prototypical SMART; Hypothesis test is that there is no difference in means at the end of the study � 1 − ρ 2 � 2 4 ( z 1 − α/ 2 + z 1 − β ) N ≥ × (2 − r ) × δ 2 where r = response rate after stage 1 treatment ρ = within-person correlation in outcome r = ρ = δ = 1 / 2 , α = . 05, need N = 142 for 80% power. Same question using a standard 2-arm trial requires N = 96. D&I Adaptive Implementation Interventions Dec 2019 54 / 118

Sample Size Formulae for Cluster-Randomized SMARTs For comparing 2 embedded AIs that begin with different treatments in a prototypical SMART 2 4 ( z 1 − α/ 2 + z 1 − β ) N ≥ × (2 − r ) × (1 + ( m − 1) ρ ) m ∗ δ 2 where r = response rate after stage 1 treatment m = avg number of SPs at each school ρ = inter-school correlation in outcome D&I Adaptive Implementation Interventions Dec 2019 55 / 118

Sample Size Formulae for Cluster-Randomized SMARTs For comparing 2 embedded AIs that begin with different treatments in a prototypical SMART 2 4 ( z 1 − α/ 2 + z 1 − β ) N ≥ × (2 − r ) × (1 + ( m − 1) ρ ) m ∗ δ 2 where r = response rate after stage 1 treatment m = avg number of SPs at each school ρ = inter-school correlation in outcome r = δ = 1 / 2 , ρ = . 03 , m = 20 , α = . 05, need N = 100 for 80%power. Same question using standard 2-arm cluster trial requires N = 66. D&I Adaptive Implementation Interventions Dec 2019 55 / 118

Recall the Autism SMART D&I Adaptive Implementation Interventions Dec 2019 56 / 118

Recall the Results Comparing the 3 DTRs Adaptive (a) TSCU (b) IJA Intervention AUC 95% CI AUC 95%CI (AAC,AAC+) 51.7 [43, 60] 9.5 [7.2,11.8] (JASP,AAC) 36.0 [28, 44] 7.2 [5.6,8.8] (JASP,JASP+) 33.1 [25, 42] 6.6 [5,8.2] No diff null p < 0 . 01 p < 0 . 05 D&I Adaptive Implementation Interventions Dec 2019 57 / 118

Recall the Example Marginal Mean Model for Longitudinal Outcomes Y t : Total Socially Communicative Utterances at t = 0 , 12 , 24 , 36. An example is the following piece-wise linear model for E [ Y t ( a 1 , a 2 ) | X ] : µ i , ( a 1 , a 2 ) ( X i ; β, η ) = β 0 + η T X + 1 t ≤ 12 { β 1 t + β 2 ta 1 } + 1 t > 12 { 12 β 1 + 12 β 2 a 1 + β 3 ( t − 12) + β 4 ( t − 12) a 1 + β 5 ( t − 12) a 1 a 2 } where X ’s are mean-centered baseline covariates. Respects the fact that some embedded must AIs share trajectories up to the point of randomization. Other marginal mean models are possible, of course! D&I Adaptive Implementation Interventions Dec 2019 58 / 118

An Estimating Equation � n � 0 = 1 − 1 I i , ( a 1 , a 2 ) ˙ µ ( X i ) V i ( a 1 , a 2 ) W i ( Y i − µ i , ( a 1 , a 2 ) ( X i ; β, η )) , n i =1 ( a 1 , a 2 ) Y i : observed longitudinal outcomes, e.g., ( Y i , 0 , Y i , 12 , Y i , 24 , Y i , 36 ) T µ i mean traj. under adaptive intervention ( a 1 , a 2 ) conditnl on X i ; � � T ∂ µ i ( X i , ( a 1 , a 2 ); β,η ) µ ( X i ) : the design matrix, i.e. ˙ ∂ ( β,η ) T V i : working cov matrix for Y i under adaptive intervention ( a 1 , a 2 ). I i , ( a 1 , a 2 ) : indicator that person i has data consistent with adaptive intervention ( a 1 , a 2 ) (this is a function of ( A 1 i , R i , A 2 i )) W i : diagonal matrix of the product of the inverse prob. of the observed treatment (this is a function of ( A 1 i , R i , A 2 i )) = (1 / Pr ( A 1 | X )) × (1 / Pr ( A 2 | X , A 1 , R )) (notation hack) next slide gives intuition for the weights D&I Adaptive Implementation Interventions Dec 2019 59 / 118

Estimation Intuition RE the weights W D&I Adaptive Implementation Interventions Dec 2019 60 / 118

An Estimating Equation � n � 0 = 1 − 1 I i , ( a 1 , a 2 ) ˙ µ ( X i ) V i ( a 1 , a 2 ) W i ( Y i − µ i , ( a 1 , a 2 ) ( X i ; β, η )) , n i =1 ( a 1 , a 2 ) Y i : observed longitudinal outcomes, e.g., ( Y i , 0 , Y i , 12 , Y i , 24 , Y i , 36 ) T µ i mean traj. under adaptive intervention ( a 1 , a 2 ) conditnl on X i ; � � T ∂ µ i ( X i , ( a 1 , a 2 ); β,η ) µ ( X i ) : the design matrix, i.e. ˙ ∂ ( β,η ) T V i : working cov matrix for Y i under adaptive intervention ( a 1 , a 2 ). I i , ( a 1 , a 2 ) : indicator that person i has data consistent with adaptive intervention ( a 1 , a 2 ) (this is a function of ( A 1 i , R i , A 2 i )) W i : diagonal matrix of the product of the inverse prob. of the observed treatment (this is a function of ( A 1 i , R i , A 2 i )) = (1 / Pr ( A 1 | X )) × (1 / Pr ( A 2 | X , A 1 , R )) (notation hack) in the weights, why not use product of inverse-probs up to t D&I Adaptive Implementation Interventions Dec 2019 61 / 118

For example, in the autism study, why not use     1 0 0 0 w i 0 0 0     0 2 0 0 0 w i 0 0 ˜     W i =  instead of W i =  where   0 0 w i 0 0 0 w i 0 0 0 0 0 0 0 w i w i w i = 2 I { A 1 = 1 , R = 1 } + 2 I { A 1 = − 1 } + 4 I { A 1 = 1 , R = 0 } ? D&I Adaptive Implementation Interventions Dec 2019 62 / 118

For example, in the autism study, why not use     1 0 0 0 0 0 0 w i     0 2 0 0 0 w i 0 0 ˜     W i =  instead of W i =  ?   0 0 0 0 0 0 w i w i 0 0 0 w i 0 0 0 w i in the weights, why use the product of the inverse probabilities? answer: with ˜ W i weights and non-diagonal V i , the cross-product terms in the estimating equations do not, in general, have mean zero D&I Adaptive Implementation Interventions Dec 2019 63 / 118

For example, in the autism study, why not use     1 0 0 0 0 0 0 w i     0 2 0 0 0 w i 0 0 ˜     W i =  instead of W i =  ?   0 0 0 0 0 0 w i w i 0 0 0 w i 0 0 0 w i in the weights, why use the product of the inverse probabilities? answer: with ˜ W i weights and non-diagonal V i , the cross-product terms in the estimating equations do not, in general, have mean zero Pepe, M. and Anderson G. (1994) A cautionary note on inference for marginal regression models with longitudinal data and general correlated response data. Communications in Statistics 23 (4), 939-951 Vansteelandt, S. (2007). On confounding, prediction and efficiency in the analysis of longitudinal and cross-sectional clustered data. Scandinavian Journal of Statistics 34 (3), 478-98. Tchetgen Tchetgen, E. J., M. M. Glymour, J. Weuve, and J. Robins (2012). Specifying the correlation structure in inverse-probability-weighting estimation for repeated measures. Epidemiology 23 (4), 644-46. D&I Adaptive Implementation Interventions Dec 2019 63 / 118

For example, in the autism study, why not use     1 0 0 0 w i 0 0 0     0 2 0 0 0 0 0 w i     ˜ W i =  instead of W i =  ?   0 0 w i 0 0 0 w i 0 0 0 0 w i 0 0 0 w i in the weights, why use the product of the inverse probabilities? answer: with ˜ W i weights and non-diagonal V i , the cross-product terms in the estimating equations do not, in general, have mean zero Using W i resolves this but not a good idea if Lots of decision points as in a micro-randomized trial; or Observational study settings where probabilities might be close to zero (multiplying the inverse of many near zero quantities will lead to very large weights) But these are not a concern in most SMARTs designed today. D&I Adaptive Implementation Interventions Dec 2019 64 / 118

Some Open Problems I’d Like to Work on Next That is, my next methods grant(s) We are wrapping up Linear Mixed Models for comparing embedded DTRs Clustered SMART with a longitudinal outcome; “3 level analysis” Multi-level SMARTs ◮ Including designs that address spillover effects R-conditional modeling and estimation to back out E [ Y t ( a 1 , a 2 ) | X ] ◮ Using a Structural Nested Mean Modeling approach How best to elicit stakeholder conjectures about optimal DTR (tailoring variables, class of decision-rules): Graphical approaches? Clinical vignettes? Some smaller, but very interesting, lower hanging fruit: small sample adjustments to sandwich standard errors, estimating the weights for efficiency (guidance on choosing covariates for this), ... D&I Adaptive Implementation Interventions Dec 2019 65 / 118

Shifting Gears a Bit: Why Mixed-Effect Models? Indirect, yet intuitive, approach to posing working models for the marginal variance-covariance of Y i = ⇒ statistical efficiency Greater flexibility in choice of working var-cov models for designs with irregularly timed measurement occasions *** Secondary interest in (1) model-based predictions of the outcome trajectories (as opposed to noisier trajectories based on the individual’s observed repeated measures) and (2) some of the variance components D&I Adaptive Implementation Interventions Dec 2019 66 / 118

A Mixed-Effect Model for the Embedded DTRs Random Intercept Only Y ( a 1 , a 2 ) = µ ( a 1 , a 2 ) + e ( a 1 , a 2 ) = β T X ( a 1 , a 2 ) + b i + ǫ ( a 1 , a 2 ) it it it i , t it where, for example, [ Y ( a 1 , a 2 ) | b i ] ∼ N ( µ ( a 1 , a 2 ) + (1 , 1 , 1 , 1) T b i , ν 2 ǫ I 4 × 4 ) i i [ b i ] ∼ N (0 , ν 2 b ) which implies [ Y ( a 1 , a 2 ) ] ∼ N ( µ ( a 1 , a 2 ) , ν 2 b (1 , 1 , 1 , 1) T (1 , 1 , 1 , 1) + ν 2 ǫ I 4 × 4 ) i i or, if σ 2 = ν 2 b /σ 2 , the marginal variance of Y ( a 1 , a 2 ) b + ν 2 ǫ , ρ = ν 2 is i   1 ρ ρ ρ   ρ 1 ρ ρ   V ( a 1 , a 2 ) = σ 2  = σ 2 C ρ  σ,ρ ρ ρ 1 ρ ρ ρ ρ 1 D&I Adaptive Implementation Interventions Dec 2019 67 / 118

A Mixed-Effect Model for the Embedded DTRs Random Intercept Only Y ( a 1 , a 2 ) = µ ( a 1 , a 2 ) + e ( a 1 , a 2 ) = β T X ( a 1 , a 2 ) + b i + ǫ ( a 1 , a 2 ) it it it i , t it where, for example, [ Y ( a 1 , a 2 ) | b i ] ∼ N ( µ ( a 1 , a 2 ) + (1 , 1 , 1 , 1) T b i , ν 2 ǫ I 4 × 4 ) i i [ b i ] ∼ N (0 , ν 2 b ) [ Y ( a 1 , a 2 ) ] ∼ N ( µ ( a 1 , a 2 ) , σ 2 C ρ ) i i But we cannot maximize the marginal log-likelihood � n f β,σ 2 ,ρ ( Y ( a 1 , a 2 ) log ˜ ) i i because we do not observe Y ( a 1 , a 2 ) . i D&I Adaptive Implementation Interventions Dec 2019 68 / 118

A Mixed-Effect Model for the Embedded DTRs Random Intercept Only Y ( a 1 , a 2 ) = µ ( a 1 , a 2 ) + e ( a 1 , a 2 ) = β T X ( a 1 , a 2 ) + b i + ǫ ( a 1 , a 2 ) it it it i , t it where, for example, [ Y ( a 1 , a 2 ) | b i ] ∼ N ( µ ( a 1 , a 2 ) + (1 , 1 , 1 , 1) T b i , ν 2 ǫ I 4 × 4 ) i i [ b i ] ∼ N (0 , ν 2 b ) [ Y ( a 1 , a 2 ) ] ∼ N ( µ ( a 1 , a 2 ) , σ 2 C ρ ) i i But we cannot maximize the marginal log-likelihood � n f β,σ 2 ,ρ ( Y ( a 1 , a 2 ) log ˜ ) i i because we do not observe Y ( a 1 , a 2 ) . i D&I Adaptive Implementation Interventions Dec 2019 69 / 118

A Mixed-Effect Model for the Embedded DTRs Random Intercept Only Y ( a 1 , a 2 ) = µ ( a 1 , a 2 ) + e ( a 1 , a 2 ) = β T X ( a 1 , a 2 ) + b i + ǫ ( a 1 , a 2 ) it it it i , t it So we propose to maximize a “pseudo log-likelihood” instead � n � ˜ W i I i , ( a 1 , a 2 ) log ˜ l β,σ 2 ,ρ ( Y i ) = f β,σ 2 ,ρ ( Y i ) i ( a 1 , a 2 ) where Y i is the observed longitudinal outcome, which leads to these estimating equations for β n � � 0 = 1 µ ( X i ) C − 1 I i , ( a 1 , a 2 ) ˙ ρ W i ( Y i − µ i , ( a 1 , a 2 ) ( X i ; β, η )) . n i =1 ( a 1 , a 2 ) D&I Adaptive Implementation Interventions Dec 2019 70 / 118

A Mixed-Effect Model for the Embedded DTRs Random Intercept Only Y ( a 1 , a 2 ) = µ ( a 1 , a 2 ) + e ( a 1 , a 2 ) = β T X ( a 1 , a 2 ) + b i + ǫ ( a 1 , a 2 ) i , t it it it it [ b i | Y ( a 1 , a 2 ) ] ∼ Normal with i � � Y ( a 1 , a 2 ) − β T X ( a 1 , a 2 ) posterior mean = ρ (1 , 1 , 1 , 1) C − 1 ρ i i , t But, again, we do not have Y ( a 1 , a 2 ) for each person! So we propose i � ˆ W i I i , ( a 1 , a 2 ) log ˜ b i = arg max f ( b i | Y i ) b i a 1 , a 2 � � �� � Y i − β T X ( a 1 , a 2 ) ρ (1 , 1 , 1 , 1) C − 1 a 1 , a 2 W i I i , ( a 1 , a 2 ) ρ i , t � = a 1 , a 2 W i I i , ( a 1 , a 2 ) D&I Adaptive Implementation Interventions Dec 2019 71 / 118

Extra, Back-pocket Slides; Some More Technical D&I Adaptive Implementation Interventions Dec 2019 73 / 118

Estimation But, first, let’s review the observed data... D&I Adaptive Implementation Interventions Dec 2019 74 / 118

An Estimating Equation � n � 0 = 1 − 1 I i , ( a 1 , a 2 ) ˙ µ ( X i ) V i ( a 1 , a 2 ) W i ( Y i − µ i , ( a 1 , a 2 ) ( X i ; β, η )) , n i =1 ( a 1 , a 2 ) Y i : observed longitudinal outcomes, e.g., ( Y i , 0 , Y i , 12 , Y i , 24 , Y i , 36 ) T D&I Adaptive Implementation Interventions Dec 2019 75 / 118

An Estimating Equation � n � 0 = 1 − 1 I i , ( a 1 , a 2 ) ˙ µ ( X i ) V i ( a 1 , a 2 ) W i ( Y i − µ i , ( a 1 , a 2 ) ( X i ; β, η )) , n i =1 ( a 1 , a 2 ) Y i : observed longitudinal outcomes, e.g., ( Y i , 0 , Y i , 12 , Y i , 24 , Y i , 36 ) T µ i mean traj. under adaptive intervention ( a 1 , a 2 ) conditnl on X i ; D&I Adaptive Implementation Interventions Dec 2019 75 / 118

An Estimating Equation � n � 0 = 1 − 1 I i , ( a 1 , a 2 ) ˙ µ ( X i ) V i ( a 1 , a 2 ) W i ( Y i − µ i , ( a 1 , a 2 ) ( X i ; β, η )) , n i =1 ( a 1 , a 2 ) Y i : observed longitudinal outcomes, e.g., ( Y i , 0 , Y i , 12 , Y i , 24 , Y i , 36 ) T µ i mean traj. under adaptive intervention ( a 1 , a 2 ) conditnl on X i ; � � T ∂ µ i ( X i , ( a 1 , a 2 ); β,η ) µ ( X i ) : the design matrix, i.e. ˙ ∂ ( β,η ) T D&I Adaptive Implementation Interventions Dec 2019 75 / 118

An Estimating Equation � n � 0 = 1 − 1 I i , ( a 1 , a 2 ) ˙ µ ( X i ) V i ( a 1 , a 2 ) W i ( Y i − µ i , ( a 1 , a 2 ) ( X i ; β, η )) , n i =1 ( a 1 , a 2 ) Y i : observed longitudinal outcomes, e.g., ( Y i , 0 , Y i , 12 , Y i , 24 , Y i , 36 ) T µ i mean traj. under adaptive intervention ( a 1 , a 2 ) conditnl on X i ; � � T ∂ µ i ( X i , ( a 1 , a 2 ); β,η ) µ ( X i ) : the design matrix, i.e. ˙ ∂ ( β,η ) T V i : working cov matrix for Y i under adaptive intervention ( a 1 , a 2 ). D&I Adaptive Implementation Interventions Dec 2019 75 / 118

An Estimating Equation � n � 0 = 1 − 1 I i , ( a 1 , a 2 ) ˙ µ ( X i ) V i ( a 1 , a 2 ) W i ( Y i − µ i , ( a 1 , a 2 ) ( X i ; β, η )) , n i =1 ( a 1 , a 2 ) Y i : observed longitudinal outcomes, e.g., ( Y i , 0 , Y i , 12 , Y i , 24 , Y i , 36 ) T µ i mean traj. under adaptive intervention ( a 1 , a 2 ) conditnl on X i ; � � T ∂ µ i ( X i , ( a 1 , a 2 ); β,η ) µ ( X i ) : the design matrix, i.e. ˙ ∂ ( β,η ) T V i : working cov matrix for Y i under adaptive intervention ( a 1 , a 2 ). I i , ( a 1 , a 2 ) : indicator that person i has data consistent with adaptive intervention ( a 1 , a 2 ) (this is a function of ( A 1 i , R i , A 2 i )) D&I Adaptive Implementation Interventions Dec 2019 75 / 118

An Estimating Equation � n � 0 = 1 − 1 I i , ( a 1 , a 2 ) ˙ µ ( X i ) V i ( a 1 , a 2 ) W i ( Y i − µ i , ( a 1 , a 2 ) ( X i ; β, η )) , n i =1 ( a 1 , a 2 ) Y i : observed longitudinal outcomes, e.g., ( Y i , 0 , Y i , 12 , Y i , 24 , Y i , 36 ) T µ i mean traj. under adaptive intervention ( a 1 , a 2 ) conditnl on X i ; � � T ∂ µ i ( X i , ( a 1 , a 2 ); β,η ) µ ( X i ) : the design matrix, i.e. ˙ ∂ ( β,η ) T V i : working cov matrix for Y i under adaptive intervention ( a 1 , a 2 ). I i , ( a 1 , a 2 ) : indicator that person i has data consistent with adaptive intervention ( a 1 , a 2 ) (this is a function of ( A 1 i , R i , A 2 i )) W i : diagonal matrix of the product of the inverse prob. of the observed treatment in stages 1 and 2 (this is a function of ( A 1 i , R i , A 2 i )) next slide gives intuition for the weights D&I Adaptive Implementation Interventions Dec 2019 75 / 118

Estimation Intuition RE the weights W D&I Adaptive Implementation Interventions Dec 2019 76 / 118

An Estimating Equation � n � 0 = 1 − 1 I i , ( a 1 , a 2 ) ˙ µ ( X i ) V i ( a 1 , a 2 ) W i ( Y i − µ i , ( a 1 , a 2 ) ( X i ; β, η )) , n i =1 ( a 1 , a 2 ) Y i : observed longitudinal outcomes, e.g., ( Y i , 0 , Y i , 12 , Y i , 24 , Y i , 36 ) T µ i mean traj. under adaptive intervention ( a 1 , a 2 ) conditnl on X i ; � � T ∂ µ i ( X i , ( a 1 , a 2 ); β,η ) µ ( X i ) : the design matrix, i.e. ˙ ∂ ( β,η ) T V i : working cov matrix for Y i under adaptive intervention ( a 1 , a 2 ). I i , ( a 1 , a 2 ) : indicator that person i has data consistent with adaptive intervention ( a 1 , a 2 ) (this is a function of ( A 1 i , R i , A 2 i )) W i : diagonal matrix of the product of the inverse prob. of the observed treatment in stages 1 and 2 (this is a function of ( A 1 i , R i , A 2 i )) in the weights, why use the product of the inverse probabilities? D&I Adaptive Implementation Interventions Dec 2019 77 / 118

For example, in the autism study, why not use     1 0 0 0 w i 0 0 0     0 2 0 0 0 w i 0 0 ˜     W i =  instead of W i =  where   0 0 w i 0 0 0 w i 0 0 0 0 0 0 0 w i w i w i = 2 I { A 1 = 1 , R = 1 } + 2 I { A 1 = − 1 } + 4 I { A 1 = 1 , R = 0 } ? D&I Adaptive Implementation Interventions Dec 2019 78 / 118

An Estimating Equation n � � 0 = 1 − 1 I i , ( a 1 , a 2 ) ˙ µ ( X i ) V i ( a 1 , a 2 ) W i ( Y i − µ i , ( a 1 , a 2 ) ( X i ; β, η )) , n i =1 ( a 1 , a 2 ) Y i : observed longitudinal outcomes, e.g., ( Y i , 0 , Y i , 12 , Y i , 24 , Y i , 36 ) T µ i mean traj. under adaptive intervention ( a 1 , a 2 ) conditnl on X i ; � � T ∂ µ i ( X i , ( a 1 , a 2 ); β,η ) µ ( X i ) : the design matrix, i.e. ˙ ∂ ( β,η ) T V i : working cov matrix for Y i under adaptive intervention ( a 1 , a 2 ). I i , ( a 1 , a 2 ) : indicator that person i is consistent with ( a 1 , a 2 ) W i : weight matrix, a function of the inverse prob. of the observed treatment in stages 1 and 2 in the weights, why use the product of the inverse probabilities? answer: because of the non-diagonal V i D&I Adaptive Implementation Interventions Dec 2019 79 / 118

You may be wondering about sample size/power?

Special Issue in Journal of Clinical Child and Adolescent Psychology APA’s Division 53 Journal Adaptive Interventions in Child and Adolescent Mental Health Editors: Daniel Almirall and Andrea Chronis-Tuscano Topics: Over 10 blinded, externally peer-reviewed papers covering anxiety, depression, autism, prevention, ADHD, child obesity Discussion: Dr. Joel Sherrill, NIMH Division of Services and Interventions Research, NIMH D&I Adaptive Implementation Interventions Dec 2019 81 / 118

SMART Case Study #4: Adaptive Implementation of Effective Programs (ADEPT) D&I Adaptive Implementation Interventions Dec 2019 82 / 118

Adaptive Implementation Intervention in Mental Health PI: Kilbourne; Co-I: Almirall (CO/AR/MI; Aim is to improve uptake of psychosocial intervention for mood disorders; primary aim compared initial REP+EF vs REP+EF+IF.) Non-responding site if: < 50% of previously identified patients were offered at least three LG sessions ( ≥ 3 out of 6)

Estimation Intuition RE the I and W D&I Adaptive Implementation Interventions Dec 2019 84 / 118

Definition of an Adaptive Intervention, in symbols { S 1 , a 1 , S 2 ( a 1 ) , a 2 , . . . , S T (¯ a T − 1 ) , a T } S t is the state or status of the individual/unit at time t and a t indexes a possible action (treatment) at time t ◮ e.g., intensify medication dose? ◮ e.g., add medication to behavioral intervention? ◮ e.g., continue treatment and monitor? An adaptive intervention is a sequence of decision rules { d 1 ( s 1 ) , d 2 ( s 1 , a 1 , s 2 ) , . . . , d T (¯ a T − 1 , ¯ s T ) } . D&I Adaptive Implementation Interventions Dec 2019 85 / 118

Interventions for Minimally Verbal Children with Autism PIs: Kasari(UCLA), Almirall(Mich), Kaiser(Vanderbilt), Smith(Rochester), Lord(Cornell)

Primary and Secondary Aims Primary Aim: What is the best first-stage treatment in terms of spoken communication at week 24: JASP vs DTT? (Sized N = 192 for this aim; compares A+B+C+D vs E+F+G+H) Secondary Aim 1: Determine whether adding a parent training provides additional benefit among children who demonstrate a positive early response to either JASP or DTT (D+H vs C+G). Secondary Aim 2: Determine whether adding JASP+DTT provides additional benefit among children who demonstrate a slow early response to either JASP or DTT (A+E vs B+F). Secondary Aim 3: Compare eight pre-specified adaptive interventions. [Note, we can now compare always JASP vs always DTT!] D&I Adaptive Implementation Interventions Dec 2019 87 / 118

Challenges in the Conduct of this Initial Autism SMART Slow responder rate, while based on prior data, was lower than anticipated during the design of the trial. Responder/Slow-responder measure could be improved to make more useful in actual practice. There was some disconnect with the definition of slow-response status and the therapist’s clinical judgment. D&I Adaptive Implementation Interventions Dec 2019 88 / 118

A Simple Regression Model for Comparing the Embedded AIs Y ( a 1 , a 2 ) denotes SCU at Wk 24 under AI ( a 1 , a 2 ). X ’s are mean-centered baseline (pre-txt) covariates. Consider the following marginal model: E [ Y ( a 1 , a 2 ) | X ] = β 0 + η T X + β 1 a 1 + β 2 I ( a 1 = 1) a 2 D&I Adaptive Implementation Interventions Dec 2019 89 / 118

A Simple Regression Model for Comparing the Embedded AIs Y ( a 1 , a 2 ) denotes SCU at Wk 24 under AI ( a 1 , a 2 ). X ’s are mean-centered baseline (pre-txt) covariates. Consider the following marginal model: E [ Y ( a 1 , a 2 ) | X ] = β 0 + η T X + β 1 a 1 + β 2 I ( a 1 = 1) a 2 E [ Y (1 , 1)] = β 0 + β 1 + β 2 = (JASP,JASP+) E [ Y (1 , − 1)] = β 0 + β 1 − β 2 = (JASP,AAC) E [ Y ( − 1 , . )] = β 0 − β 1 = (AAC,AAC+) D&I Adaptive Implementation Interventions Dec 2019 89 / 118

A Simple Regression Model for Comparing the Embedded AIs Y ( a 1 , a 2 ) denotes SCU at Wk 24 under AI ( a 1 , a 2 ). X ’s are mean-centered baseline (pre-txt) covariates. Consider the following marginal model: E [ Y ( a 1 , a 2 ) | X ] = β 0 + η T X + β 1 a 1 + β 2 I ( a 1 = 1) a 2 − 2 β 1 + β 2 = (AAC,AAC+) vs (JASP,JASP+) − 2 β 1 − β 2 = (AAC,AAC+) vs (JASP,AAC) − 2 β 2 = (JASP,AAC) vs (JASP,JASP+) D&I Adaptive Implementation Interventions Dec 2019 90 / 118