Fidelity to the EBDP Programs in Michigan: Findings of Evaluation - PowerPoint PPT Presentation

APPENDIX F3 Fidelity to the EBDP Programs in Michigan: Findings of Evaluation Report Joan Ilardo, PhD, LMSW Assistant Professor, Director of Research Training. Core Faculty-Geriatric Education Center of Michigan

Purpose of PATH Fidelity Assessment • Determine degree to which sample of PATH leaders adhere to PATH curriculum

Study Design: Addressed 5 Questions – Completed PATH leader training – Co-led PATH workshop – Conducted PATH leader and master trainer online survey in winter 2010-2011 (118 responses) – Conducted 6 key informant interviews of PATH coordinators, master trainers and T-trainers – Conducted 3 site observations (sessions 2 and 5)

Evaluation Question 1 • How are PATH leaders reflecting on what occurs in sessions they conduct? Do they conduct peer or self evaluations?

Evaluation Question Results • 2/3 of PATH Leaders used recommended self-evaluation form after conducting workshop. • 86% talk with co-leaders after each session to discuss how it went • Another 11% have a discussion at end of workshop series • Only 3% reported they never discussed how things went at PATH sessions.

Evaluation Question 2 • What are the PATH leaders’ perceptions of the leader training?

Evaluation Question 2 Results • Felt trainers did well explaining PATH was an evidence-based curriculum based on Stanford Chronic Disease Self-Management Program • Aware of research showing positive results for participants who apply what they learn. • 93% reported master trainers did very well emphasizing importance of “sticking to the book”. • All interviewees were emphatic that curriculum is closely followed.

Evaluation Question 3 • Are there ways PATH leader support could be enhanced to increase fidelity?

Evaluation Question 3 Results • Different versions of the leader manual • Guided imagery -Session 5, what to do if a participant does not want to do the activity • Action Planning – How to do them? – Writing them down? • No consensus on leaders doing self- evaluations

Evaluation Question 4 • What are the strengths and challenges encountered by PATH leaders as they conduct classes?

Evaluation Question 4 Results • Strengths – Understand concept of fidelity and work to impart that in leader training – Recruit leaders who are familiar with PATH and see how program can benefit participants – Very conscious about sticking to book and did excellent job – Stayed on topic – Set good boundaries while being approachable – Strong role models: engaged, positive, respectful, caring

Evaluation Question 4 Results Challenges Noted by respondents • Participants’ literacy levels • Frail, elderly groups • Quiet groups • Chatty groups • Diverse groups • Dominating individuals • Self-interested individuals

Evaluation Question 4 Results Challenges Observed • Positive thinking activity • Writing letter to provider • Buddy call

Evaluation Question 5 • What ways can data gathered by OSA and MDCH be used to monitor PATH fidelity?

Evaluation Question 5 Results • All coordinators review information on MI PATH forms before sending them in • Use data from MDCH for – Reports to agencies – Recruiting participants – Presentations to medical groups – Publicity – Feedback for PATH leaders • No system in place to track outcomes but would welcome the information

Recommendations • Guidance from MDCH about resources available at PATH workshops • Recruit lay leaders who have chronic conditions since participants identify with them. • To increase fidelity, all leaders should role model dealing with difficult emotions in Session 2. • Booster sessions should be developed by MDCH and OSA that reinforce best practices for the action plan and problem-solving activities. • Session 5 closing activity is difficult because of the various strategies offered to participants. Clarification could be provided through the booster session. • Session 5 Positive Thinking activity can be confusing. Stanford could be approached about alternative ways to practice this concept such as role modeling by co-leaders.

Recommendations • Systems for connecting leaders who prefer to work together could increase leader retention and satisfaction. • Coordinator should be trained about how to develop/nurture relationships with primary care oriented-organizations to reinforce the benefits of PATH for patients with chronic diseases and ways to encourage patients to attend. • One way to engage physicians is to offer to conduct behavior change follow-up surveys for their patients who attend PATH workshops. Follow-up surveys could be conducted at 6 and 12 months. • Develop relationships with residency and fellowship to include evidence-based self-management programs as part of their community resources segment of fellowship training. • Consider using a Facebook page as an additional way to promote MI PATH. The official MI PATH website link could be part of the page.