Understanding Engagement in Research Laura Forsythe, PhD Lori - PowerPoint PPT Presentation

Understanding Engagement in Research Laura Forsythe, PhD Lori Frank, PhD Victoria Szydlowski Evaluation & Analysis

PCORI Evaluation Framework

Objectives for Measuring Engagement • Describe engagement in PCORI funded projects • Evaluate impact on PCORI strategic goals • Inform PCORI funding requirements • Guide current awardees, future applicants, and others interested in PCOR • Support project progress based on learnings

Ways of Engaging - ENgagement ACtivity Tool (WE-ENACT) Pilot project awardees: • Baseline and Project End • Subsequent awardees: Baseline and Y ear 1 • Awardees nominate research partners to be invited to respond

WE-ENACT Data Collection Researchers N Patient/stakeholder (% response rate) partners N (% response rate) Baseline 60 (82%) 97 (54%) Year 1 99 (71%) 177 (56%) End of project (Pilot projects) 27 (54%) 25 (56%) TOTAL 186 299

Methods for Qualitative Analysis (N= 105 researchers, 93 patients and stakeholders) • Partnership with American Institutes for Research • Developed & applied codebook based on research questions and review of the open-text responses • Identified major themes Mapped themes to conceptual model of PCOR 1 • 1 Frank L, Forsythe L, Ellis L, Schrandt S, Sheridan S, Gerson J, Konopka K, Daugherty S. Conceptual and practical foundations of patient engagement in research at the patient-centered outcomes research institute. Qual Life Res. 2015 Jan 6.

Qualitative Research Questions • Engagement strategies • Barriers and facilitators • Impact of engagement • Differences by respondent type • Questionnaire improvements

Data analyzed by content codes Focus on codes with >25 responses • engagement strategies • engagement impact • barriers • facilitators • how stakeholders got involved in the project • relationships • knowledge/training • logistical issues • PCOR principles

Conceptual Model of PCOR ACTIONS Initiate and maintain • OUTCOMES partnerships FOUNDATIONAL • Near-term • Communicate OUTCOMES ELEMENTS • Intermediate • Capture and use • N O e U ar T -t C O m MES er Long-term • perspectives • Inte mediate near- • Readiness Ensure influence • L o n eg - term I n t rm • Channels • • Train te r m e d iate long- • • Share term PCOR Principles Frank L, Forsythe L, Ellis L, Schrandt S, Sheridan S, Gerson J, Konopka K, Daugherty S. Conceptual and Practical Foundations of Patient Engagement in Research at the Patient-Centered Outcomes Research Institute Plans. Quality of Life Research 2015. 24(5):1033-1041 DOI 10.1007/s11136-014-0893-3

Results: Approaches to Engagement Y ear 1 Researcher Report (N=90) Co-Investigator Research T eam Member Advisory Group Group Forum Surveys 0 .1 .2 .3 .4 .5 .6 .7 .8 .9 1 Percent of Projects

Results: Stages of Engagement Y ear 1 Researcher Report (N=91) Identifying ResearchT opics Developing Research Questions Proposal Development Developing the Budget Adding people to the team Study Design Recruiting Participants Data Collection DataAnalysis Results Interpretation Dissemination 0 .1 .2 .3 .4 .5 .6 .7 .8 .9 1

Results: T ype of Stakeholders Engaged Y ear 1 Researcher Report (N=91) 59 Patient 51 Clinician 31 Health System Advocacy Organization 28 Caregiver 27 Policy Maker 10 Payer 7 6 Training Institution 4 Purchaser 4 Life Sciences Industry 0 10 20 30 40 50 60 70 80 90 100 Percent of Projects

Conceptual Model of PCOR ACTIONS Initiate and maintain • OUTCOMES partnerships FOUNDATIONAL • Near-term • Communicate OUTCOMES ELEMENTS • Intermediate • Capture and use • N O e U ar T -t C O m MES er Long-term • perspectives • Inte mediate near- • Readiness Ensure influence • L o n eg - term I n t rm • Channels • • Train te r m e d iate long- • • Share term PCOR Principles

Foundational Elements & Training: Institutional Review Board (IRB) • Researchers noted challenges with IRB communications: • Categorizing patient/stakeholder partners • Involving protected or special populations as partners • Explaining IRB process to patient/stakeholder partners • Several researchers noted the need for human subjects protections training for patient/stakeholder partners Stakeholders don't always understand members are not research subjects the confines of IRBs, contractual but are not scientific co-investigators agreements and ethical responsibilities. (thus, they shouldn't have to do 6 hour of computer module ethics and study design training to be able to be a "co-investigator" on the project).

Foundational Elements: Compensation Many respondents noted the importance of compensating patients and • stakeholders for their time • Some researchers expressed concerns about: – balancing appropriate compensation vs. coercion – unintended adverse consequences (e.g., eligibility for social programs) – discrepancy between physician and patient In all honesty , the amount I am being paid We are concerned about ($100/hour) really helps me engage because it how compensation affects makes me so willing to help in any way possible, eligibility for public as well as offer ideas on how I can help. benefits, including Medicaid, welfare, and food stamps. …At what level do you compensate patients without it seeming like they are being co-opted?

Conceptual Model of PCOR ACTIONS Initiate and maintain • OUTCOMES partnerships FOUNDATIONAL • Near-term • Communicate OUTCOMES ELEMENTS • Intermediate • Capture and use • N O e U ar T -t C O m MES er Long-term • perspectives • Inte mediate near- • Readiness Ensure influence • L o n eg - term I n t rm • Channels • • Train te r m e d iate long- • • Share term PCOR Principles

Results: Initiate and Maintain Partnerships • Patients and stakeholders noted the usefulness of being involved early or experienced a desire to be involved earlier • Researchers noted several challenges: keeping patients engaged throughout the project, setting expectations for project funding, and lacking funds for early involvement “I wish they would have contacted us “It is always hard to go back to earlier in the grant process so we may stakeholders…when a project has not have been able to work in more areas been funded. This, in my opinion, is of the state vs. a small section. ” one of the greatest challenges to engaging with patients in the conceptualization and planning “We did not have money to phases. ” reimburse patients/stakeholders as we prepared the grant. ”

Conceptual Model of PCOR ACTIONS Initiate and maintain • OUTCOMES partnerships FOUNDATIONAL • Near-term • Communicate OUTCOMES ELEMENTS • Intermediate • Capture and use • N O e U ar T -t C O m MES er Long-term • perspectives • Inte mediate near- • Readiness Ensure influence • L o n eg - term I n t rm • Channels • • Train te r m e d iate long- • • Share term PCOR Principles

Results: Communication • Managing power differentials • Managing diverse groups • Using plain language “Researchers need to understand patients and how to communicate “How does one facilitate op w i f t r h t t h r e m , r e e s f p e r c ia ll I y if they are not in in as uc tu o D & >> conversations across the same age group or cultural different stakeholder background. ” groups when there are strong feelings that can conflict? ” “It is sometimes difficult to "speak the same language" at group meetings. In other words, the language style tends to be dominated by researchers or clinicians. ”

Conceptual Model of PCOR ACTIONS Initiate and maintain • OUTCOMES partnerships FOUNDATIONAL • Near-term • Communicate OUTCOMES ELEMENTS • Intermediate • Capture and use • N O e U ar T -t C O m MES er Long-term • perspectives • Inte mediate near- • Readiness Ensure influence • L o n eg - term I n t rm • Channels • • Train te r m e d iate long- • • Share term PCOR Principles

Capturing the Patient Perspective: Study Design • Help choose research methods: measures, interventions, comparators, and outcomes • Decide on data collection methods • Review and revise study plans and materials “We presented aspects of study “Helped the investigators design to the group and solicited decide what cohort of their input. I.e. - what should the patients to include. ” "control group" be for the RCT - is it ‘standard practice’ … or should it be a currently existing but potentially “Gave clinical input into choice ineffective intervention. ” of screening measures. ”