How Organizational Factors Influence Training Impact Over Time - - PowerPoint PPT Presentation

How Organizational Factors Influence Training Impact Over Time within Community Mental Health Clinics

Victoria Stanhope, Abigail Ross, Mimi Choy-Brown, & Lauren Jessell

Acknowledgements

Investigators Victoria Stanhope (PI), NYU Steven Marcus, Penn Larry Davidson, Yale PRCH Janis Tondora, Yale PRCH Deborah Padgett, NYU

PCCP Study Team

Diane Grieder, AliPar, Inc Maria Restrepo-Toro, Yale PRCH Stephanie Lanteri, Yale PRCH Mimi Choy-Brown, NYU Lauren Jessell, NYU Taylor Kravitz, NYU Lynden Bond, NYU Abigail Ross, Fordham University Liz Matthews, Rutgers University Meredith Doherty, Hunter College This work was supported by NIMH R01MH099012 Person-Centered Care Planning and Service Engagement

The content is solely the responsibility of the authors and does not necessarily represent the views of the National Institutes of Health

Mandate to Improve Service Quality

“Care that is respectful of and responsive to individual patient preferences, needs and values and ensuring those values guide all clinical decisions”

An Emerging EBP: Person-Centered Care Planning

» 2-DAY training with expert trainers » 12-months of follow-up technical assistance calls twice/month with expert trainers targeting supervisors as trainers and addressing challenges

5

Community Mental Health Clinics (N=14) with 273 providers Intervention N=7 Control N=7

Implementation Strategy for Person-Centered Care Planning

Implementation Readiness

• Multi-dimensional concept & critical for translation
• Change Commitment
• Change Efficacy
• Schein’s primary mechanisms for embedding a new practice
• What leaders pay attention to on a regular basis shape provider

priorities

• How they signal their interest & support of an effort

Role of Implementation Leadership

Proactive Supportive Knowledgeable Perseverant

Study Aim & Design

• A longitudinal mixed-methods study examined a one-year training process

across seven community mental health clinics to explore the relationship between organizational factors and implementation readiness over time

• Embedded in a randomized controlled trial of Person-Centered Care

Planning in community mental health clinics

• Mixed-methods design
• Sequential Explanatory
• Focus groups utilized to explain trainer ratings of the training process

and observed leadership participation in the training process

Quantitative Data Sources

• Trainer Ratings:
• Following the 2 monthly TA calls per site (one supervisor call, one

team-based call), trainers completed assessments for each of 7 experimental sites

• Proactive Leadership (single item 1-10 likert)
• Implementation Readiness (single item 1-10 likert)
• Observed TA Call Attendance Ratios:
• Attendance recorded for each TA call (n=24)
• Ratio calculated by: (# attending participants) / (# eligible

participants) for leadership and supervisor subgroups

Quantitative Analyses

• Pearson’s r correlation were used to examine the relationship between

proactive leadership and implementation readiness

• Means and standard deviations were computed for Phase 1(Months 1-6)

and Phase 2 (Months 7-12) of 12 month training period

• Paired sample t-tests were used to examine changes between Phases 1

and 2 of training on outcomes of proactive leadership, implementation readiness, and observed participation

Characteristic Overall (N=157) Direct Care (n=87) Supervisors (n=45) Leadership (n=25) Gender Female N=110(70.5%) N=61 (70.5%) N=33 (73.3%) N=16 (64.0%) Male N=46 (29.5%) N=25 (29.1%) N=12 (26.7%) N=9 (36.0%) Age 43.3 (SD=12.4) 40.1 (SD=11.9) 43.6 (SD=11.0) 53.73 (SD=10.7) Race/Ethnicity White N=96 (61.1%) N=47 (53.0%) 26 (57.8%) 23 (92.0%) Non-White N=61 (38.9%) N=36 (41.4%) 17 (37.8%) 2 (4.0%) Education HS/Some college/AA N=19 (12.0%)

N=15 (17.2%)

N=3 (6.7%) N=1 (4.0%) Undergraduate N=60 (38.2%)

N=40 (46.0%) N=14 (31.1%) N=6 (24.0%) Graduate N=77 (49.0%) N=31 (35.6%) N=38 (62.2%) N=17 (72.0%) Time at agency

Less than 1 year 24 (15.5%) N=17 (19.5%) N=4 (8.9%) N=3 (12.0%) 1-3 years 41 (26.1%) N=27 (31.0%) N=13 (28.9%) N=5 (20.0%) 3 or more years 92 (58.6%) N=41 (47.1%) N=28 (62.2%) N=17 (68.0%)

Demographics

1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 11 12

Proactive Leadership and Implementation Readiness By Month

Implementation Readiness Proactive Leadership

r=0.69

Month of TA Call Trainer Rating

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1 2 3 4 5 6 7 8 9 10 11 12

Trainer Ratings and Observed Leadership Participation by Month

Implementation Readiness (Rescaled) Proactive Leadership (Rescaled) Observed Leadership Participation Ratio

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1 2 3 4 5 6 7 8 9 10 11 12

Trainer Ratings and Observed Leadership Participation on Supervisor Calls by Month

Implementation Readiness (Rescaled) Proactive Leadership (Rescaled) Supervisor Call Leadership Participation Ratio

Differences Over 12 Month Training Period

Phase 1 (M1-M6): M(SD) Phase 2 (M7-M12): M(SD) P t (df) Trainer Ratings Proactive Leadership 5.39(2.21) 6.74(1.74) *0.03 2.73(6) Implementation Readiness 4.52(2.78) 5.94(2.01) *0.04 2.63(6) Observed Participation Ratios Supervisors Participation 0.51(0.15) 0.64(0.16) *0.02 3.10(6) Leadership Participation 0.45(0.32) 0.74(0.21) *0.03 2.98(6)

Qualitative Methods

• Focus groups of supervisors and direct care staff were conducted at the 7

experimental sites during rolling 12-month intervention period

• 104 participants total, 8-12 participants per focus group
• One hour groups led by two masters-level interviewers
• Topical domains included: perceptions of PCCP, experiences with PCCP

training, and barriers and facilitators to implementation

Qualitative Analysis

• Focus groups were digitally recorded, transcribed verbatim and uploaded

into Atlas TI for analysis

• Using thematic analysis (Boyatzis, 1998), three researchers co-coded the

15 transcripts, generating categorical codes and an initial codebook reflecting patterns in the data

• A final codebook was generated through consensus and codes specific to

the training process were extracted and collapsed into key themes

• Strategies for rigor included weekly team debriefings and the use of an

audit trail

Qualitative Findings

• Four themes emerged
• Already Doing it
• Resistance
• Top Down
• The Ah-Ha Moment

Already Doing It

• I mean, we’ve just, we’ve done it, it’s just now it’s put a title to it of, okay, this is

exactly what it’s called that we do. Um, I feel like it, it was very in line with how we did our jobs previously (DC/Site 2)

• I mean, I feel kind of like, um—of course [pause] most of, you know, if you ask

nine out of ten clinicians, of course they’re going to be, like, “I’m really happy with the way things are going in our program, we don’t need to change anything.” (SUP/Site 7)

• Um, so I definitely felt the staff were open but a lot of them, again, folk who had

been here a little bit longer, it was kind of like, “How is this different again? Or are we just calling it a different name?” (SUP/Site 6)

Resistance

• I think in the overall what I have to do and juggle and balance, I don’t think—I don’t

think everyone’s on the same page that that is the best method. Um, so again, it’s a lot of pushback of us to like everybody else, like, “No, this isn’t the way it should be.” So it’s a lot of fighting with—as you go up and up……You know, because if this is something that [the state] is requiring, you can resist it and you can not like it, but it’s going to happen. (DC/Site 2)

• The other thing that I remember about the first email… said, “You’ve been chosen

for this great opportunity,” and we’re all going, “Oh, no.” But she tried really hard to frame it as something really good. But there was moaning and groaning. (SUP/Site 3)

Top Down

• No, he just told us, “Do it.” [laughter] “I want you to do it. You’re going to do it.”

(SUP/Site 2)

• Definitely my supervisor would have done it in supervision, kind of like, “Oh, by the

way, we’re doing this thing.” (DC/Site 3)

• You just, you just muscle through it. It’s just, you know, you can complain all you

want but at the end of the day, it’s got to get done, so. (DC/Site 8)

• But am I understanding what it—uh, the, the purpose of all of it and how it’s—no.

I’m just doing it. [laughs] But I’m not really comprehending the importance of it. (DC/Site 2)

The Ah-ha Moment

• Got an email that we were doing this and I remember the reaction of the people was,

“Person-centered treatment plans? We always did do person-centered treatment plans. We don’t need to learn that.” Come to find out it is a different, it is a little bit of a different process. (SUP/Site 3)

• I think, um, for my program, I think there’s not—I think person-centered, again, like I said, it

was very delayed, my response to it, but it’s been very positive and I don’t think it’s something that’s going to fall by the wayside. (SUP/Site 6)

• Um, the other thing is, is I’ve been doing treatment plans for a, a long time, so it was, it was

kinda—initially it wasn’t—but towards the end it got a little refreshing, um, because you—I got caught in, in—this is the way I do it. (SUP/Site 7)

• And it wasn’t just the same old stuff, either, like DMHS trains us, and it was actually fun

activities and getting the staff to think about it and getting that ah-ha moment for them because of the material that we had. (SUP/Site 8)

Discussion

• The significant increase in implementation readiness in the second six

months of training compared to first six months suggests providers were in pre-contemplation phase for first half of training

• Lack of increase in implementation readiness in the first six of training was

explained by the belief they were already doing PCCP, resistance to a new practice and minimal buy-in

• Leadership behavior was closely related to implementation readiness
• Leaders played a directive role but were not proactive in terms of their

visibility and involvement in the first six months

Limitations

• Unidimensional measure for implementation readiness
• Implementation readiness reliant on trainers’ perceptions
• Focus groups subject to social desirability
• No measure of implementation outcomes

Implications

• Results suggest training impact could be increased and dose reduced

through the utilization of the following implementation strategies

• Assess the stage of change of providers and address resistance

prior to training

• Involve providers in decision-making to implement PCCP in order to

increase buy-in

• Engaging leadership in training process from the outset