Best Practices for Independent Peer Reviews Jacqueline Patterson - PowerPoint PPT Presentation

Toxicology Excellence for Risk Assessment Best Practices for Independent Peer Reviews Jacqueline Patterson Toxicology Excellence for Risk Assessment (TERA) Society for Risk Analysis (SRA) Annual Meeting December 11, 2013 Independent ● Non-Profit ● Science for Public Health Protection

Robust Processes To insure quality and independence of results • Scientific – Selection of experts – Quality and completeness of review materials – Comprehensive charge questions – Clear reporting of results • Administrative – Peer review plan – Clear instructions to experts – Defined roles for participants – Documentation of results – Independence Independent ● Non-Profit ● Science 2 for Public Health Protection

Principles or Best Practices for a Successful Peer Review • Appropriate Experts • Scientific Robustness • Transparency • Independence Independent ● Non-Profit ● Science 3 for Public Health Protection

Appropriate Independent Experts • Appropriate experience to address key issues • Balance of views on key topics • Diversity in institutional backgrounds and Peer– a person perspectives who is the equal • Risk assessment expertise of another in abilities and essential qualifications Independent ● Non-Profit ● Science 4 for Public Health Protection

Panel Selection Procedures • Evaluate subject document closely to identify key questions, guiding determination of types of experts and experience needed • Broadly search for experts • Evaluate credentials and potential for conflicts of interest and biases • Select a panel that is balanced with regard to necessary disciplines and has a diversity of perspectives Independent ● Non-Profit ● Science 5 for Public Health Protection

Peer Review for Risk Assessment – Data – Alternate interpretations – Mode of action – Assumptions – Calculations – Counterfactual evidence – Conclusions Independent ● Non-Profit ● Science 6 for Public Health Protection

Complete Review Package • Quality work product – Ready for Review • Balance between essential information and reviewers’ time • References and data readily available • Clear charge and instructions • Assist peer reviewers Independent ● Non-Profit ● Science 7 for Public Health Protection

Peer Review Charge Purpose - identify the key scientific issues and guide the experts to consider each important issue in a multi- disciplined way • Comprehensive - yet focused on most important questions to address purpose of peer review • Experts must understand the purpose and context for the assessment – problem formulation • Objective - developed independently- with input from authors – to insure all key issues included and objectivity of question wording • Provide experts clearly worded questions, with well- defined criteria for evaluation and clear instructions Independent ● Non-Profit ● Science 8 for Public Health Protection

Panel Meetings Skilled chair with risk assessment expertise • to guide panel through issues Focused agenda and pacing • Authors present to answer and ask • questions Interested parties allowed to contribute • scientific data and analyses and ask questions Independent ● Non-Profit ● Science 9 for Public Health Protection

Transparency • An operational philosophy that allows interested persons to evaluate and judge the adequacy and credibility of the process and results • Open meetings/process that welcomes input • Documentation of process and results • Independent evaluation of responsiveness of authors Independent ● Non-Profit ● Science 10 for Public Health Protection

Independence – more than just conflict of interest • Process and participants need to be independent of the outcome • A third party – not the authors or sponsoring agency or organization, should - – Select experts – Prepare charge questions – Keep authors, sponsors, and stakeholders at arm’s length from experts • Evaluate authors’ responsiveness Independent ● Non-Profit ● Science 11 for Public Health Protection

The most critical element Independent ● Non-Profit ● Science 12 for Public Health Protection

Peer Review is Important • Recent survey by George Mason University of SRA, SOT, and ISRTP risk assessors found that*: • 73% agreed that external independent peer review is very important • 78% indicated external peer review should be conducted independently of the office or program that develops a risk assessment • 65% would create an independent entity that insures authors would respond to peer review comments *Robert Lichter presentation, Webinar and Study Report available at www.istrp.org Independent ● Non-Profit ● Science 13 for Public Health Protection

What do peer reviewers think about current peer review practices? • What is essential for a good peer review? • What hinders a good peer review? • What improvements or changes would you suggest? Independent ● Non-Profit ● Science 14 for Public Health Protection

What do peer reviewers think about current peer review practices? • What is essential for a good peer review? • What hinders a good peer review? • What improvements or changes would you suggest? Independent ● Non-Profit ● Science 15 for Public Health Protection

References • Meek, ME, Patterson, J; Strawson, J; Liteplo, R. (2007) Engaging Expert Peers in the Development of Risk Assessments. Risk Anal. 27(6):1609- 1621. • Turner, RS. 2009. Best Practices in Peer Review Assure Quality, Value, Objectivity. Journal of the National Grants Management Association 17:1 • George Mason University. 2013. Expert Opinion on Regulatory Risk Assessment. Results of a Survey of SRA, SOT/RASS and ISRTP members . Available at www.Isrtp.org Independent ● Non-Profit ● Science 16 for Public Health Protection