CIHR INTERNATIONAL PEER REVIEW EXPERT PANEL UNIVERSITY DELEGATE NETWORK REPRESENTATIVE: JENNIFER J. McGRATH, UDEC, Concordia Univ. JANUARY 17, 2017 Allocated 5 minutes to present during first morning session, of Panel’s second day of FIVE MINUTE OBJECTIVES 1) Early Milestones of Reforms meetings. Presented summary of ongoing UD Network discussions and compiled information from online survey that sought specific input to original questions posed by 2) Seven Concerns Raised Repeatedly Panel. Four other representatives from CIHR research community also presented during this session (Drs. Jim Woodgett, Holly Witteman, Kristen Connor, Michael Hendricks). Dr. McGrath is currently one of the longest Two objectives of presentation: (i) highlight early milestones in evolution of reforms, and standing UD Network members (since April (ii) identify seven points repeatedly raised by science community. Following presentations, 2009), making her well-positioned to identify representatives answered follow-up questions raised by Expert Panel. critical points in development of reforms. ORIGINAL QUESTIONS POSED BY PANEL INTERNATIONAL PANEL 1. Does design of CIHR’s reforms of investigator-initiated programs & peer review –Chair: Sir Peter Gluckman, Chief Scientific processes address their original objectives? Advisor to Prime Minister, New Zealand 2. Do changes in program architecture and peer review allow CIHR to address challenges –Dr. Trish Groves, Director Academic posed by breadth of its mandate, evolving nature of science, and growth of Outreach & Advocacy, British Medical Journal interdisciplinary science? –Professor Mats Ulfendahl, former Secretary- 3. What challenges in adjudication of applications for funding have been identified for General for Medicine & Health at Swedish public funding agencies internationally and in literature on peer review and how do Research Council CIHR’s reforms address these? –Professor Mark Ferguson, Director General, Science Foundation Ireland & Chief Scientific 4. Are mechanisms set up by CIHR, including but not limited to College of Reviewers Adviser to the Government of Ireland appropriate and sufficient to ensure peer review quality and impacts? –Professor Jonathan Grant, Director of the 5. What are international best practices in peer review that should be considered by Policy Institute, Assistant Principal for Strategy, CIHR to enhance quality and efficiency of its systems? King’s College London, UK 6. What are leading indicators and methods through which CIHR could evaluate quality –Dr. Michael Lauer, Deputy Director, Office and efficiency of its peer review systems going forward? of Extramural Research, National Institutes of Health, USA –Professor Dame Anne Glover, Vice-Principal External Affairs & Dean for Europe, University of Aberdeen EARLY MILESTONES IN EVOLUTION OF REFORMS [History repeats itself] ● HEALTH RESEARCH ROADMAP 2009 –Alain Beaudet CIHR’s Strategic Plan 2009-10 – 2013-4 Strategic Direction 1: “Our peer review system has been internationally recognized for its design and effectiveness….we will make improvements to the system where they are needed, while also building on its strengths.” (p 15) “CIHR will continue to ensure that the peer review system is able to meet the knowledge requirements of health researchers across all four pillars…we will strengthen our processes and criteria for identifying excellence and innovation so that proposals from each pillar of health research are evaluated with the same degree of rigour and fairness.” (p 15) ● UNIVERSITY DELEGATE FACE-TO-FACE MEETING June 2009 –Alain Beaudet “CIHR will ensure transparency and accountability” CIHR’s University Delegate Meeting Preliminary Results Web Survey Results; Web based survey launched April June 4, 2009 2009 (n=330; 71% academic sector): “94-77% support [or strongly support] Slides #19-25 Strategic Direction 1 – World Class Excellence” “Explore creating a more flexible committee structure through a pool of –Greg Huyer, Peer Review Management Unit expert reviewers ‘on standby’ who are called to review depending on the Update on Peer Review Management scope of applications received → Help to reduce reviewer fatigue and the Activities last-minute scramble for reviewers” June 4, 2009 Slides #12, 13 “For the OGP, consider pooling related committees and splitting only after the applications have been received (cf. NSERC ‘conference’ model) → Allow for a proactive, rather than a reactive, response to changes in application Page 1 of 6
scope and pressure” “Ensure that external reviewers are used only when truly necessary.” Ensure necessary expertise present in “Structure programs and committees to reduce the need for external the room reviewers, by ensuring as much as possible that the necessary expertise can be present in the room.” ● SURVEY OF STAKEHOLDER SATISFACTION IN PEER REVIEW –Ipsos Reid (2010) survey conducted for 2010 CIHR’s 2011 International Review [Funding Success Rate in 2010: 23%] ● DESIGN DISCUSSION DOCUMENT 2011 –Draft Version, Design Discussion Document Version 1.0 – December 16, 2011 “Availability of expertise is a critical component in CIHR’s peer review process…growing need for CIHR to recruit peer reviewers from a broader bases of expertise to ensure all aspects and future impacts of health research are considered…it is becoming increasingly difficult to populate panels with enough breadth of expertise to fully meet the requirements of emerging and multidisciplinary areas of research.” (p 17) “There is disagreement on what should be funded 25% of the time, consistent with other studies of peer review…The reliability and consistency of peer review decisions are critical components of CIHR’s selection process.” (p 18) “As Mayo and colleagues point out: ‘Despite science’s pre-occupation with “79% of peer reviewers are satisfied accurate measurement, there is no precise method measuring the quality of with the efficiency of the peer review proposals…’ ” (p 19) process, while 70% of peer reviewers [Referring to Ipsos Survey Figure above:] “while CIHR’s peer reviewers find are satisfied with the overall fairness the process to be fair and effective, there is room for improvement” (p 19) of the process.” (p 19) ● UNIVERSITY DELEGATE FACE-TO-FACE MEETING – CIHR’s University Delegate Meeting June “June 27, 2011” [Correct date: June 2] 2011 “Since release of Designing for the Future document, a number of Slides #5 & 6 presentations have been made…60 Townhalls…institution administrators…professional societies…national roundtables…advisory [Bolded emphasis retained from original.] groups…Some common themes have emerged…” (p 5) “1. Why is CIHR changing the programs and the peer review system at the same time ? This is a lot of change at once and is risky.” “2. Research is done in the four pillars in very different ways. Why are you using the same adjudication criteria for all pillars ?” “4. How will you ensure that one pillar is not negatively impacted by the change?” “6. My grant is ending during the transition period and I will have a gap in funding . What are my options?” “8. Will there be enough reviewers to adjudicate applications to the Page 2 of 6
Recommend
More recommend
