Therapist-Reported Competence: A Pragmatic Fidelity Measurement Strategy for Integration into Community Practice Settings Rochelle F. Hanson 1 , Benjamin E. Saunders 1 , Jason Chapman 2 Sonja Schoenwald 1 , Angela Moreland 1 1 Medical University of South Carolina 2 Oregon Social Learning Center Presentation at the 9 th Annual Conference on the Science of Dissemination & Implementation, December 14, 2016, Washington DC
Acknowledgements » Duke Endowment (No. 1582-SP) » NIMH (Grant No. R34 MH104470-02)
Colleagues Medical University of South Carolina Benjamin E. Saunders, PhD Elizabeth Ralston, PhD Angie Moreland, PhD Elizabeth Hinson, MSW Heidi Resnick, PhD Rachael Garrett, MSW Michael de Arellano, PhD Carole Swiecicki, PhD Dan Smith, PhD Kim Reese, MSW Dee Norton Lowcountry Children’s Jan Koenig, MEd Center Faraday Davies Sara delMas Monica Fitzgerald, PhD Emily Fanguy University of Colorado
Intro: Measuring Fidelity » Observational coding methods » Expensive, time consuming, generally not feasible & sustainable in usual care/community-based settings » Potential alternative: therapist self-report Fidelity: Adherence vs. Competence
Focus of Presentation » To examine feasibility of therapist self-report as measure of EBT use over course of training/implementation project » To examine relationships between therapist self-reported competence of an EBT (TF-CBT) and client outcomes (i.e., predictive validity )
Participants
Targeted EBT Treatment Duration 8-20 Sessions Treatment Frequency 1 session/week
Fidelity: Prescribed Order of Components P sychoeducation Stabiliz bilization ion R elaxation 1/3 1/ 3 Phas hase e A ffective Modulation Gradual Exposure P arenting Skills C ognitive Coping Trauma auma Time 1/ 1/3 3 Narrativ Nar ive e T rauma Narrative Phas hase e and Processing Int ntegration/ ion/ I n vivo 1/3 1/ 3 Cons onsolida olidation ion C onjoint sessions Phas hase e E nhancing safety
CBLC Training Requirements for Clinical Participants » Complete pre-work, including TF-CBTWeb, and pre-CBLC evaluation » Attend all learning sessions » Participate in 12/14 calls » Complete TF-CBT with a minimum of 2 cases (register 5) » Complete weekly metrics to assess use and competence in delivery of TF-CBT with training cases » Complete monthly metrics on delivery of TF-CBT with all clients on caseload » Complete post-CBLC evaluation
Participants » Clinicians ( n = 570) who attended LS1 of a Project BEST LC or CBLC ( n = 11). 516 identified at least one training case 446 had at least one competence rating (total of 1,767 cases)
Training Cases » 2,361 identified training cases » 1,614 (68.4%) had pre or post child-reported PTS » 816 (34.6%) had pre and post child-reported PTS (Jason) u 60.2% girls u 55.3% White; 34.7% African American; 3.1% Hispanic; 7% Other u M child age = 12.34 (SD = 3.42; range = 4-23 years)
Fidelity Measurement » Weekly online reports to assess: use of and perceived competence in delivering 11 TF-CBT components » Treatment duration (specified as 16-20 sessions), » Inclusion of a caregiver, and » Prescribed order of components
Weekly Clinical Metrics • Supervision minutes • Registered cases seen Each Case • Parental involvement • Which TF-CBT Components • Perceived Competence in delivery ( less than adequate skill to expert skill ) • Barriers to adherence Collected metrics up to 24 weeks
Results: Use of TF-CBT Components » M weekly metrics completed per case = 11.26 ( SD = 4.77) 50.3% of clinicians completed at least 12 weeks of metrics 33.5% of clinicians completed at least 14 weeks of metrics » Treatment duration (# of weeks from pre to post assessment) M = 23.11 ( SD = 9.3), Mode = 19.0, Range = 6-52 sessions » Clinicians reported completing an average of 8.86 (of 11) TF- CBT components, and at least 10/11 with 50.8% of their cases All PRAC components were completed with 76% TN completed with 79.1% In Vivo completed with 44.3% Enhancing safety completed with 78.4%
Perceived Competence » 4 Competence Outcomes by Case » Minimum, maximum, first and last competence ratings » Variability in Competence Across Components Lowest ratings on exposure-based components; highest on psychoeducation, parenting skills and relaxation » Change in Perceived Competence Over Time Mixed=Effects Regression Models [Clients (level-1) nested within Therapists (level-2)] Change over time: # of months between first Competence rating for any client and the first Competence rating for each subsequent client
Growth Models: Change in Competence Over Time
TF-CBT Training Cases: PTSD with pre/post PTSD assessments completed (n = 816) UCLA CPSS ( n = 297) ( n =519) Pre Post Pre Post Mean 35.1 19.0 25.9 12.0 SD 13.4 11.9 10.3 9.4 d 1.3 1.4 t t (296)=19.8* t (518)=28.8* * p <.001 Recent RCT Results : Cohen et al. (2011) pre-post child UCLA total: d = 0.64 Deblinger et al. (2011) mean pre-post for child outcomes: d = 0.94
TF-CBT Training Cases Scoring Above Clinical Cut Score [N = 816] Percent 100 CPSS ≥ 15 80 UCLA ≥ 38 60 40 71% 20 24% 0 Pre-Tx Post-Tx X 2 (1) = 54.5, p < .001
Therapist Perceived Competence & Child Pre to Post Tx PTS Outcomes » Analyses Three-level model with pre-post measurements (level-1) nested within clients (level-2) nested within therapists (level-3). Therapist competence scores entered as predictors at client level. To test for change, a dummy coded indicator was included at level-1 to differentiate the post-assessments from the pre- assessments The model made use of all available data
Therapist Perceived Competence and Child Pre to Post Tx Outcome (PTS) Those who received services from therapists with higher self-rated competence had greater improvements in PTS symptoms pre-to-post treatment
Discussion » Variability in self-reported competence – lowest on exposure-based components, highest on PRAC » Significant improvements on 4 of 11 components (P, R, A, Conjoint) » Trend for TN development and Enhancing Safety » Children who received tx from therapists with higher self-rated competence had greater improvements in PTS sx pre to post treatment. » Potential for low cost, low burden, feasible measure of competence
Discussion: Study Limitations » No control condition (selection bias; limited generalizability; regression to the mean…) » No objective fidelity coding for comparison with therapist self-report (i.e., is TF-CBT cause of reduction in symptoms??) » Therapist rating style
Next Steps » Ongoing data cleaning, cleaning and more cleaning » Therapist variables related to fidelity (e.g., therapist rating style) » Changes in adherence as related to implementation stages » Convergent validity » So many questions……
Questions?
Contact Information Rochelle Hanson hansonrf@musc.edu
Recommend
More recommend
