New NIH requirements regarding Rigor and Reproducibility - PowerPoint PPT Presentation

New NIH requirements regarding Rigor and Reproducibility http://grants.nih.gov/reproducibility/faqs.htm 1. The scientific premise of the proposed research 2. Rigorous experimental design for robust and unbiased results 3. Consideration of relevant biological variables 4. Authentication of key biological and/or chemical resources https://grants.nih.gov/reproducibility/module_1/presentation.html https://www.nigms.nih.gov/training/pages/clearinghouse-for-training- modules-to-enhance-data-reproducibility.aspx

1. The scientific premise of the proposed research Addressed in “research strategy” Reviewed as part of “significance” and “overall impact” 2. Rigorous experimental design for robust and unbiased results Addressed in “research strategy” Reviewed as part of “approach” and “overall impact” 3. Consideration of relevant biological variables Addressed in “research strategy” Reviewed as part of “approach” and “overall impact” 4. Authentication of key biological and/or chemical resources IF RELEVANT, upload a plan as an “other attachment” (FORMS-C before May 24, FORMS-D after May 25, 2016) Not reviewable; reviewers will comment on this plan but will not score it

4. Authentication of key biological and/or chemical resources Briefly describe methods to ensure the identity and validity of key biological and/or chemical resources used in the proposed studies Key biological and/or chemical resources may or may not be generated with NIH funds and: 1) may differ from laboratory to laboratory or over time; 2) may have qualities and/or qualifications that could influence the research data; and 3) are integral to the proposed research. These include, but are not limited to, cell lines, specialty chemicals, antibodies, and other biologics. Standard laboratory reagents that are not expected to vary do not need to be included in the plan. Examples are buffers and other common biologicals or chemicals. NIH funds may be used to authenticate resources Save this information in a single file named “Authentication of Key Resources Plan,” and attach it as Item 12 on the Other Project Information form.

1. The scientific premise of the proposed research ( research plan ) Applicants must state the scientific premise that forms the basis for the proposed research question(s). NIH expects applicants to describe the general strengths and weaknesses of prior research cited by the applicant, including the rigor of previous experimental designs, their examination of relevant biological variables and their authentication of key resources 2. Rigorous experimental design for robust and unbiased results ( research plan ) Scientific rigor is the strict application of the scientific method to ensure robust and unbiased experimental design, methodology, analysis, interpretation and reporting. This requires full disclosure of experimental details so that others may reproduce and extend the findings.

3. Consideration of relevant biological variables ( human and/or animal subjects ) Biological variables , such as sex, age, weight, and underlying health conditions are often critical factors affecting health or disease. NIH expects that sex as a biological variable will be factored into research designs, analyses, and reporting in vertebrate animal and human studies. Strong justification must be provided for applications proposing to study only one sex. 4. Authentication of key biological and/or chemical resources ( other attachments ) Key biological and/or chemical resources include cell lines, specialty chemicals, antibodies and other biologics. These may differ between laboratories or over time, and may influence the research data and the ability to reproduce the results. Investigators must determine which resources used fit these criteria and are, therefore, key to the proposed research.

Scored Review Criteria Significance Is there a strong scientific premise for the project? Approach Have the investigators presented strategies to ensure a robust and unbiased approach, as appropriate for the work proposed? Have the investigators presented adequate plans to address relevant biological variables, such as sex, for studies in vertebrate animals or human subjects? Authentication of Key Biological and/or Chemical Resources For projects involving key biological and/or chemical resources, reviewers will comment on the brief plans proposed for identifying and ensuring the validity of those resources.

Phase I The first phase is designed to incorporate changes to the following four policy areas (for applications due between January 26 and May 25, 2016): Policy Area Guide Notice Quick Summary Rigor and NOT-OD-16-011 New research strategy language; new "Authentication Transparency of Key Biological and/or Chemical Resources" attachment. New review criteria. See the Guide notice for a list of exceptions. NOT-OD-16- Vertebrate Simplifies required criteria, descriptions, and 006 Animals justifications. Excludes fellowship and training grants. Definition of Child NOT-OD-16- "Child" under inclusion policy is now under 18 years old. 010 Research Training NOT-OD-16- Specifies information needed for PHS 398 Research 004 Training Program Plan.

Phase II The second phase includes the actual switch from FORMS-C to FORMS-D and incorporates the following policy changes in addition to those listed above : Policy Area Guide Notice Quick Summary Rigor and NOT-OD-16- Reproducibility changes extend to training and Transparency 011 fellowship applications; new Plan for the Instruction in Methods for Enhancing Reproducibility attachment. Additional details to follow in December. Vertebrate Animals NOT-OD-16- Changes listed above extend to training and fellowship 006 applications. Inclusion Forms TBA Adding an optional PHS Inclusion Enrollment Report form to FORMS-D application packages (with additional study descriptors). Data Safety NOT-OD-16- New Data Safety Monitoring Plan attachment required Monitoring Plans 004 for all applications involving clinical trials. NIH will validate on this form if clinical trials box checked on application. Research Training NOT-OD-16- Introduces xTRACT, an eRA Commons system for Data Tables 007 creating research training data tables and storing the reported information; reduces the number of tables from 12 to 8, minimizes individual-level information, and extends trainee outcome tracking from 10 to 15 years.

Phase II The second phase includes the actual switch from FORMS-C to FORMS-D and incorporates the following policy changes in addition to those listed above : Policy Area Guide Notice Quick Summary PHS Assignment NOT-OD-16- NIH Division of Receipt and Referral will no longer honor Request Form 008 requests for study section assignment in the cover letter. Applicants must use the new Assignment Request Form for IC assignment preference, study section preference, potential conflicts, and needed expertise. Form will be treated as current cover letter with regard to privacy and permissions. Font Guidelines NOT-OD-16- Provides additional flexibility regarding the fonts allowed 009 in PDF attachments. Biosketch NOT-OD-16- Rules for URL to publication list; cite work in both Clarifications 004 personal statement and contributions to science sections; graphics, figures, and tables are not allowed.

Key Changes Changes focus on the following areas, but implementation is divided into phases: • Rigor and transparency in research – both Phases I & II • Vertebrate animals – both Phases I & II • Definition of Child – Phase I • Requirements for PHS 398 Research Training Program Plan - Phase I • Inclusion reporting – Phase II • Data safety monitoring – Phase II • Institutional Training Grants - new tables required Phase II • Font requirements – Phase II • Biosketch clarifications – Phase II