CREDO Research Request June State Board of Education Meeting June - PowerPoint PPT Presentation

CREDO Research Request June State Board of Education Meeting June 9, 2016

About the Research  Requestor: Margaret (Macke) Raymond  Institution: CREDO at Stanford University  Study Duration: From date of approval to December 2017.  Purpose: To update a 2013 quasi-experimental study of how students in schools affiliated with Charter Management Organizations and charter networks progress academically.  Funding: The proposed research is funded by the Walton Family Foundation and the Fisher Fund. 2

Internal Vetting Process  Question: Is the research compliant with the Family Educational Rights and Privacy Act (FERPA)?  Answer: Yes. Specifically, the disclosure to CREDO would be permitted under the following FERPA exception: “The disclosure is to organizations conducting studies for, or on behalf of, educational agencies or institutions to: (A) Develop, validate, or administer predictive tests; (B) Administer student aid programs; or (C) Improve instruction.”  Furthermore, FERPA has additional requirements regarding the need for contractual agreements with the researcher that covers a number of subjects. If this research is approved, CDE will put in place a Data Sharing Agreement with CREDO that includes all these requirements. 3

Internal Vetting Process Cont.  Question: What is the validity of the research plan (is the researcher qualified)?  Answer: Dr. Raymond serves as director of the Center for Research on Education Outcomes (CREDO), which analyzes education reform efforts around the country. She has done extensive work in public policy and education reform, and is currently researching the effectiveness of public charter schools and the creation of reliable data on program performance. Prior to joining Stanford in 2000, she held faculty positions in the political science and economics departments at the University of Rochester. 4

Internal Vetting Process Cont.  Question: What is the significance of research proposal?  Answer: The aim of the research is to provide education leaders and policy makers with solid evidence about program and policy performance for their use in decision making. Since charter schools and Charter Management Organizations in particular are dynamic in size and effort, current assessment of whether and how they are effective can have immediate value in policy discussions.  Question: Is the researcher in good standing on any previous projects?  Answer: Yes. The researcher has conducted prior research for CDE and has provided us with the results of that research. An executive summary of the results of the prior research is included in your packets. 5

Internal Vetting Process Cont.  Question: Is the research of specific benefit to CDE?  Answer: There is potential to derive benefit from learning how students in charter schools perform compared to their peers in non-charter schools.  This research is a continuation of research that has been ongoing in Colorado for a number of years, and the potential to have data regarding student outcomes in charter and non-charter schools may benefit charter schools and authorizers, as well as families as part of the school choice process.  Legislators may also be interested in the findings from this research as they make policy decisions about school governance structures. 6

Internal Vetting Process Cont.  Question: Does the research align with CDE’s and the Board of Education’s strategic priorities?  Answer: Answer unknown as CDE continues to develop its updated strategic priorities. However, CREDO supports CDE in developing and meeting priorities that support the success of all Colorado children and youth. 7

Internal Vetting Process Cont.  Question: What is the validity of the research plan (are the methods appropriate)?  Answer: We use a matching methodology to compare the growth of a charter student to the typical growth they would have had in a traditional public school using the same year/grade/subject tests. The "virtual twin" method used by CREDO has been peer reviewed by other researchers 1 and found to be a reliable and valuable method of analysis. 1 Forston, K. and Verbitsky-Savitz, N. et al. (2012). “Using an Experimental Evaluation of Charter Schools to Test Whether Nonexperimental Comparison Group Methods Can Replicate Experimental Impact Estimates,” NCEE 2012-4019, U.S. Department of Education 8

Internal Vetting Process Cont.  Question: What are the data elements specified?  Answer: The following data elements have been requested: state achievement test scaled scores, proficiency categories, school ID number, race/ethnicity, gender, free- reduced lunch status, special education status, English proficiency, grade level, full student enrollment files, graduation flag, course completion records, charter schools by district and school ID, grade level means and standard deviations for the state reading and math tests, conditional standard error of measurement tables, cut scores for proficiency bands, and teacher-student linkage.  The only direct identifier is the State Assigned Student ID (SASID) which CREDO has requested be masked. 9

Internal Vetting Process Cont.  Question: Do we want to share those data elements with the researcher? Are any data elements sensitive in any way?  Answer: CDE’s data owners for each data type have approved the preceding list to be provided to the researcher.  The researcher requested a few additional data types (raw test scores and juvenile detention identification) but we will not provide that data due to its sensitivity. 10

Internal Vetting Process Cont.  Question: Has IRB approval been provided?  Answer: Yes. The IRB approval from Stanford University for this research is contained in your packets.  Question: Has the data security training documentation been provided?  Answer: Yes. CREDO provided the following information:  Any officers or employees of CREDO who have access to student data or teacher or principal data have received training on the federal law governing confidentiality of such data through the Collaborative Institutional Training Initiative (CITI) prior to receiving access to student, teacher or principal data. 11

Next Steps (Pending Approval)  CDE will work with the researcher to put in place a Research Data Sharing Agreement that includes the following:  Strict controls around how the researcher can use data.  Stringent security and privacy protections that must be in place to protect the data.  Requirements for the researcher to provide CDE with the results of the research.  Requirements for the researcher to destroy the data once the research has been completed on a schedule specifically listed in the Research Data Sharing Agreement.  Research Data Sharing Agreements are reviewed on an annual basis, should the research extend beyond a year.  Once the term of the Data Sharing Agreement has reached its end, CDE will follow up with the researcher to obtain the results of the research and gain a confirmation that the data has been destroyed. 12

Approval Request Motion 1: To approve the CREDO Research Proposal. Motion 2: To deny the CREDO Research Proposal. 13