Spring 2015 Cycle: Large Pragmatic Studies to Evaluate - PowerPoint PPT Presentation

Spring 2015 Cycle: Large Pragmatic Studies to Evaluate Patient-Centered Outcomes Applicant Town Hall June 10, 2015

Your letter of intent (LOI) was reviewed and you have been invited to submit a full application…….. CONGRATULATIONS!

Agenda Welcome Overview Merit Review Criteria Other topics What to think about before you apply Submit questions via the Questions? chat function in Meeting Bridge. Ask a question via phone (an operator will standby to take your questions).

Overview Overview David Hickam, MD, MPH Program Director Clinical Effectiveness Research

Purpose of the Pragmatic Studies PFA To assist applicants to prepare strong proposals in response to this funding announcement: Applicants proposing “clinical comparative effectiveness research (CER)” Applicants proposing “improving healthcare systems (IHS) CER” Applicants proposing “CER to reduce or eliminate health and health care disparities”

What is a Pragmatic CER Study? Answers a practical, real world comparative effectiveness research question. Assesses whether two or more options differ in effectiveness when administered as they are in real life Project is conducted in a clinical setting that is as close as possible to a real world setting. The methodological approach (including study design, outcome measures, and follow-up) is as simple as possible without sacrificing scientific rigor.

What is a Pragmatic CER Study? (Cont.) Pragmatic CER trials addressing populations at risk for disparities may:  Require higher levels of outcome data collection.  Require tailoring of intervention (e.g., language, culture, access).  Consist of multi-component, multi-level interventions (e.g., targeting the patient, provider, and systems), as evidenced by disparities literature.

Justification for the Design Elements of a Large Pragmatic Study Suggest reviewing pragmatic–explanatory continuum indicator summary (PRECIS) tool Consider tradeoffs  Eligibility criteria  Flexibility of intervention  Range and types of outcomes  Follow up intensity  Adherence  Etc. Source: A pragmatic–explanatory continuum indicator summary (PRECIS): a tool to help trial designers. Thorpe, et al. CMAJ 2009; 180:E47-E57.

Research Activities Not Supported in this PFA Studies of decision aids Efficacy trials Evidence syntheses Cost-effectiveness analysis Research that aims to compare the overall costs of care between two or more alternatives and use the results to determine the preferred alternative

Merit Review Criteria Impact and Potential to Improve Care Steven Clauser, PhD, MPA Program Director Improving Healthcare Systems

Merit Review Criteria Impact of the condition on health of individuals and populations Potential for the study’s results to improve care and outcomes Technical merit Patient-centeredness Engagement with key patient (including caregiver), clinician, and/or other stakeholder organizations

Impact of the Condition In the US population, or in underserved sub- populations: Is the condition or disease associated with a significant burden - in terms of prevalence, mortality, morbidity, costs to society, individual suffering, or loss of productivity?

When the Condition is a Healthcare System Problem In the US population, or in underserved sub- populations: Is the healthcare system problem associated with poor access to care, suboptimal care quality, high burden to society or to healthcare systems, individual suffering, mortality, morbidity, and/or loss of productivity?

Addressing Impact of the Condition or System Problem Clinical CER  State the prevalence of the condition  Discuss the impact of the condition on the individual Improving Healthcare Systems CER  Describe the magnitude of the healthcare system problem  Discuss the impact of the system problem on healthcare access and quality – and on individual and population suffering, morbidity, mortality, productivity and costs

Potential for the Study’s Findings to Improve Care and Outcomes What knowledge gap underlies the problem? How does that gap manifest in practice? Would positive study findings close the gap and improve practice and outcomes? Would such findings be adopted widely?

Addressing Potential for Study Findings to Improve Care and Outcomes Provide evidence of an important knowledge gap:  A recent systematic review or clinical guidelines pointing to a research need. An update of previous systematic review is also acceptable. Provide evidence of substantial variation in practice, resulting from the knowledge gap, if known. Provide indications that the proposed CER has a substantial potential to improve practice and patient outcomes?  Should refer to existing efficacy or effectiveness studies  IHS CER should refer to prior evidence of efficacy or effectiveness of components of current intervention, or of effectiveness of the interventions in smaller studies or other settings

Addressing Potential (cont.) Summarize ongoing studies of the intervention Discuss the likelihood that positive study findings related to improvements in practice and patient outcomes would be implemented widely and quickly  Breadth of current use in existing practice  Extent of patient, practice or organizational barriers and facilitators to rapid adoption by patients, clinicians and relevant health organizations  Applicability to diverse delivery systems  Endorsement of the current study by key patient, physician and other stakeholder groups. Describe how the partners in your proposal (e.g., national and/or regional stakeholder organizations) would help disseminate the study’s findings to potential adopters

Merit Review Criteria Technical Merit and Methodology Standards David Hickam, MD, MPH Program Director, Clinical Effectiveness Research

Technical Merit A clear research plan with rigorous methods that adhere to PCORI’s Methodology Standards and accepted best practices A clear justification for the study design A carefully constructed and realistic timeline that includes specific scientific and engagement milestones A research team with documented appropriate expertise and experience - and a rational organizational structure An excellent research environment, including the appropriate delivery system(s) to host the study, adequate resources, and strong support of the proposed research

Addressing Technical Merit Describe how all relevant PCORI Methodology Standards are addressed Justify the proposed sample size estimates, including the anticipated effect size and standard deviation, and all other relevant parameters Describe anticipated rates of and feasibility of recruitment Justify the proposed study design Describe the validity of the proposed measures of patient outcomes Discuss proposed analytic methods, including subgroup analyses Discuss proposed approach to consent and protection of human participants

PCORI Methodology Standards 47 standards in 11 groups. The Methodology Standards do not address all issues related to study designs and methods. Note that PCORI is not using a specific set of methodological standards for “pragmatic studies.”  Consider design tradeoffs (e.g., blinding vs not blinding)  Refer to other respected sources for additional guidance.  View here: http://www.pcori.org/assets/2013/11/PCORI- Methodology-Report.pdf

Sample Size Estimate and Subgroups Document previous relevant studies to justify use of the proposed effect size. Assure appropriate adjustments based on the particular study circumstances, including the study population, expected dropout and crossover, and other parameters. Demonstrate sufficient power to analyze important pre-specified subgroups; these subgroups should be based on patient attributes with strong a priori pathophysiological or empirical justification.

Cluster Randomized Trial Design Need justification for the use of this design  State the benefits and disadvantages of a cluster design vs. individual randomization, based on the clinical issues and the interventions being studied. Sample size estimates need to account for reasonable estimates of intraclass correlation. Maximize the number of clusters that are randomized.

Merit Review Criteria Patient-Centeredness and Patient Engagement Sue Sheridan, MIM, MBA, DHL Program Director, Patient Engagement

Patient-Centeredness Is the research focused on questions that affect outcomes of interest to patient and their caregivers?  Does the research question address choices that are important to - and faced frequently by - patients, their caregivers, or clinicians?  Is the study powered on outcomes that are important to patients? Does the research address one or more of the key questions mentioned in PCORI’s definition of patient-centered outcomes research?

Addressing Patient-Centeredness Provide evidence that the research question(s) and outcomes are important to patients (and/or their caregivers)  Have you obtained the endorsement and participation of major patient/caregiver organizations that would use the research? Describe your strategy for measuring outcomes that are important to patients.