Improving Implementation with the SPeeding Research INTerventions - PowerPoint PPT Presentation

Improving Implementation with the SPeeding Research INTerventions (SPRINT) Program Anna Gaysynsky, MPH Alisa Ainbinder, PhD, MA, April Oh, PHD, MPH, Tara Loomis, MBA, MA Cynthia Vinson, PHD, MPA 10th Annual Conference on the Science of Dissemination and Implementation Arlington, VA December 5, 2017

Evaluation Approach  Two cohorts of SPRINT to date (Cohort I in Summer 2016, Cohort II in Spring of 2017)  SPRINT was evaluated using a mixed-methods approach that collected both quantitative and qualitative data from SPRINT participants (n=20 teams consisting of 2-4 team members).  Response/participation rate:  Cohort I post-course survey: 81%  Cohort I focus groups: Everyone who attended closeout session (exact number not available)  Cohort II post-course survey: 78%  Cohort II focus Groups: 85%  Presentation will cover evaluation results, barriers to commercialization, and the relationship between D&I and commercialization 2

Evaluation Results 3

Overall SPRINT Course Rating*  Cohort 1:  Cohort 2: Overall, how would you rate the SPRINT Overall, how would you rate the SPRINT course? course? 60% 60% 48% 50% 50% 45% 45% 44% 40% 40% 30% 30% 20% 20% 9% 8% 10% 10% 0% 0% 0% 0% 0% 0% Excellent Very good Fairly good Not very good Not good at all Very good Good Fair Poor Very poor * Note: response options differed slightly in the two versions of the 4 survey

Faculty Rating* “Although it was uncomfortable at times, the instructors' insistence that we challenge our assumptions was important.”  Cohort 1:  Cohort 2:  Most respondents (86%) rated interactions  Most respondents (83%) rated interactions with facilitators as “Good/ Excellent”, and felt with facilitators as “Good/Very good”, and felt their feedback was helpful (86%) their feedback was helpful (75%) Instructor Ratings Instructor Ratings 60% 60% 50% 50% 50% 50% 50% 42% 36% 36% 40% 40% 33% 33% 30% 30% 21% 17% 20% 20% 14% 9% 10% 10% 5% 4% 0% 0% Quality of interactions with faculty Feedback from faculty on ideas Quality of interactions with Feedback from facilitators on members facilitators ideas Very good Good Fair Poor Very poor Excellent Good Fair Poor Very Poor 5 * Note: response options differed slightly in the two versions of the survey

Would Participants Recommend SPRINT?  Cohort 1:  Cohort 2:  The majority of respondents (86%)  The majority of respondents (84%) would recommend the course to other would recommend the course to other researchers researchers. Would you recommend SPRINT to other eligible Would you recommend SPRINT to other researchers? eligible researchers? 100% 100% 86% 90% 84% 80% 80% 70% 60% 60% 50% 40% 40% 30% 20% 20% 14% 12% 10% 4% 0% 0% 0% Yes Unsure No Yes Unsure No 6

Would Participants Recommend SPRINT? Comments  Absolutely. We need more researchers to think like this.  Yes!! What an amazing experience that we will carry with us forever! I think anyone that gets a R01 should be required to go through this process before they begin their research work.  It should be mandatory training for new investigators.  I would only recommend SPRINT it its current form if a researcher wanted to start a business/start up . I do not think it is well suited for researchers who want to implement their interventions into real world clinical settings. Perhaps a program like TIDIRH would be better for the latter.  Not unless the grant will cover some PI time . It was very clear that PIs were to devote 20% time for the 6-7 weeks. Which is fine but needs to be built into clinical, etc duties…  The training was worthwhile but the learning experience became very stressful because of the pace. Each team member spent over 8-12 hour/week on SPRINT- related activities 7

Time Commitment  Participants spent an average of 12 hours per week on the course (median = 10)  Participants in Cohort II spent about half that time (6 hours per week) conducting interviews [Data not available for Cohort I]  In the focus groups there were some mixed feelings about the intensity of the course and what (if anything) should be done about it.  [The immersion was] painful but set the pace  Immersion was key but [we] cannot keep up that pace  Need[ed] more time for reflection  Might be helpful for initial training to be longer so that there were not “16 hour days”  But if process lasted longer, it would be hard to justify (institutionally) as well as schedule the time (personally/professionally) 8

Was the stress worth it? While many participants felt that the time commitment was challenging, in the end, most participants felt SPRINT was a valuable and worthwhile learning opportunity. I thought the SPRINT program was terrific! It was a lot of work in a short amount of time but I'm very thankful that I was chosen to participate as I learned a great deal about the current product/intervention and skills/experiences that I can incorporate into future interventions. … the course was intense and the materials were challenging . I worked a lot harder (put in more time) than I had expected. But there is no easy way to teach difficult materials; there is no easy way to push people outside of their comfort zone…I learned a great deal and I'm very grateful for the opportunity. 9

What Did Participants Get from SPRINT?*  Cohort 1:  Cohort 2: Obtained new skills/knowledge as a result of Obtained new skills/knowledge as a result of participation in SPRINT. participation in the program 100% 100% 71% 67% 50% 50% 33% 25% 4% 0% 0% 0% 0% 0% 0% 0% 0% Totally agree Agree Strongly agree Agree Neutral Neither agree nor disagree Disagree Totally disagree Doesn't apply/No response Disagree Strongly disagree Feel confident applying the skills/knowledge Feel confident applying the skills/knowledge developed during the institute developed during the course 60% 60% 52% 50% 43% 42% 40% 40% 20% 20% 8% 5% 0% 0% 0% 0% 0% 0% 0% Strongly agree Agree Totally agree Agree Neutral Disagree Neither agree nor disagree Disagree 10 Strongly disagree Totally disagree Doesn't apply

Knowledge and Skills Gained: General  [SPRINT] reshaped the way I look at interventions and delivery of services in general ... I never saw it [before] as selling a product… I look forward to applying some of things I learned in my personal research.  I learned not just things to help the intervention we have now- but interventions for the future- so definitely it’s a skill development  My dissemination plans are going to be a whole lot different now . And that’s exciting.  I’ve described it as “ learning how to think about the realities of disseminating and implementing interventions in ways that, given an academic position, you tend not to think about ”… it’s been very helpful…to learn some of those real world barriers and challenges that exist in implementing an intervention  it really gives you a way to think about business so you can avoid some of the pitfalls- because you can think through the whole process in advance… 11

Knowledge and Skills Gained: Interviews  I don’t think we necessarily knew the best way to engage or talk with … insurance companies and people who decide about business and investment... this was a good crash course in how to engage those stakeholders and figure out what questions we should be asking ... Because what information they need to make their various decisions – [is] not in our mindset.  at first we all did interviews with people in our field the first couple of times, who are just like us and had the same perspectives. But by the end of the course we were interviewing people totally outside of our fields. That’s been transformative .  learning how to network with people who were outside of the ivory tower in the real world … I was completely clueless about that and intimidated by it…[but] I enjoyed finding common ground with people that we’re going to have to have relationships with if we’re going to impact public health through the dissemination of our intervention .  just talking to people who would be using the tool we’re developing, it’s critical for making sure it actually meets the needs and demands of practitioners and patients  getting out of the building… it’s been the most valuable thing to my career so far. I’m meeting people that do the same work I do- but not researchers - different stakeholders that all have the same common goal. If it wasn’t for this program - I wouldn’t be invited to sit at some of the tables I’ve now been invited to sit at in the last 6 weeks… 12

Interview Requirement  Most teams did between 30 and 45 interviews, but a few teams did significantly more that the required minimum of 30.  Some participants felt there was too much of a focus/too much pressure on the interview requirement  I just think there’s a lot of pressure to get the interviews done …we absolutely did some interviews where our time could have been better used-but it was just that desperateness to get something in.  I felt like [instructors] were too focused on the numbers [of interviews]. I felt like I wanted to take a step back and think about, “what questions do I need to ask and get answered?” “Where do I really need to go?” “Where do I want to focus?” Whereas the push was really for numbers and just, "go do it"  Minimum number of required interviews for next cohort: 40. 13