T AKING A S TATEWIDE L ONGITUDINAL D ATA S YSTEM (SLDS) F ROM F UNDAMENTALS TO A DVANCED C APABILITIES Melissa Straw, Wisconsin DPI Brian Pritzl, VersiFit Technologies Ernie Morgan, Value Added Research Center (VARC) Mike Christian, Value Added Research Center (VARC) 25 th Annual STATS-DC 2012 Data Conference
A GENDA Agenda Item Presenter Introductions Melissa, Brian, Ernie, Mike Data and Reporting Fundamentals Melissa Introduction to Growth Ernie Advanced Analytics: SGP, WISEdash, and Melissa Guided Analysis Partnerships Melissa, Brian, Ernie, Mike Advanced Analytics: Value-Added and Brian, Ernie, Mike Edvantage Next Steps Melissa Wrap-Up and Q&A
D ATA AND R EPORTING F UNDAMENTALS
D ATA S YSTEM B ACKGROUND P RE -G RANT Implementation of an individual student enrollment system data collection in 2004-05. Included third Friday enrollment, year end indicators, discipline, and IDEA child count. Creation of a Unique Student ID (WSN) for all WI public students through Wisconsin Student Locator System (WSLS) Student data available starting with the 2005-06 school year.
SLDS G RANTS 1 ST G ENERATION D ATA P ROJECTS LDS Data Warehouse or ODS First student-centric data warehouse at DPI Stored and linked student and school data from a variety of sources including collection systems, spreadsheets, and external files for reporting and analysis ISES YE, ISES CD, ISES Discipline, WSAS, ACT, AP, SGP, ACCESS ELL, Graduation, School Data, WSLS, State Outcomes Data, P20 NSC Enrollment Data, Student Growth Percentile (SGP) Data, Graduation Cohort and Rate Data Coursework Completion System (CWCS) Course data including a student-teacher-course link
SLDS G RANTS 1 ST G ENERATION D ASHBOARD AND REPORTING P ROJECTS Dashboards and Reporting Secure WINSS SDPR WISEmaps Home MDAT LDSAM/ASM MDAT SAFE Training
SLDS G RANTS N EXT G ENERATION D ATA , D ASHBOARDS , R EPORTING DPI purchased a suite of data, dashboard and reporting tools Data Model & Dashboard from VersiFit Technologies Based in Appleton Serves MMSD, MPS, Chicago Public Schools, Oregon State, Hawaii, etc. Focus on business intelligence solutions specific to education Contract signed 02/16/11 ETL and Reporting tools from Microsoft Reporting tool will also be utilized throughout the agency for operational reporting. The LDS ODS is now currently being used as a data store as we transition to our new data warehouse system. Wisconsin Information System for Education Dashboard solution or WISEdash, powered by Edvantage
WISE DASH A RCHITECTURE Dashboards ETL DW/LDS Users & Reports Clean Conform Create Collections Statewide State Assessments P20/NSC Value Added SGP Licensure ECE Data SIS Students Schools 8
E DVANTAGE ™ D OMAINS Alumni Test Actions Graduate Participation Outcomes Program Test Scores Metrics Student-Staff Membership Support Associations Schedules Financial Balances Special Instruction Education Budgets Practices Student Staff Marks Absences Student Program Cohorts Attendance Student Performance Housing Impact Purchasing, Diploma Accounts Requirements Payable, Student Inventory Health & Medical Students, Staff Conditions Schools, Effectiveness Data Calendars, Quality Accounts, Staff Qualifications Engagement Staff & Development Staff Recruitment & Evaluation Transport Enrollment School/ Snapshots Surveys Department Highly Metrics Qualified Behavior Mobility Enrollment Stability Financial Period Daily Aid Attendance Attendance At-Risk Recruitment Students Truancy 9
SLDS G RANTS D ASHBOARDS AND REPORTING Dashboards and Reporting Public Reporting Performance WINSS or Report Cards WISEmaps Secure Home WISEdash (SDPR) (future) ASM MDAT MDAT Training SAFE WISEdash
I NTRODUCTION TO G ROWTH
A CHIEVEMENT , G ROWTH AND V ALUE -A DDED Achievement (NCLB) a “point in time” measure of student proficiency compares the measured proficiency rate with a predefined proficiency goal Gain (MDAT) measures average gain in student scores from one year to the next Growth (SGPs) measures average gain in student scores from one year to the next accounts for the prior knowledge of students Valued-Added (VARC) measures contribution of teachers/schools/districts to gain in student scores accounts for the prior knowledge of students accounts for student demographic characteristics accounts for test measurement error
A LL THESE MEASURES ARE BASED ON STUDENT TEST DATA , BUT ANALYZED DIFFERENT WAYS Attainment Gain/Growth Value-Added Evaluate based on student Yes Yes Yes testing performance Evaluate schools for meeting the No Yes Yes needs of all students Hold schools accountable for No No Yes what they can control
C OMBINING M EASURERS : A CHIEVEMENT , G ROWTH OR V ALUE -A DDED For the most complete picture of student and school performance, it is best to look at Achievement in combination with Growth or Value-Added. This will tell you: What students know at a point in time ( Achievement ) How students at your school are growing ( Growth ) or how your school is affecting student academic growth ( Value-Added )
A DVANCED A NALYTICS : SGP, WISE DASH , AND G UIDED A NALYSIS
D EMONSTRATION Tutorial WISEdash SGP visualization Guided Analysis Note: The tutorial shown in the presentation will soon be available here: http://dpi.wi.gov/lds/dashhome.html.
P ARTNERSHIPS
E VOLUTION T IMELINE Statewide Value-Added Opt-In Year 1 (advisory group) Madison Milwaukee Metropolitan Statewide Opt- Public School In Year 2 Schools District Statewide Statewide All Demonstration Districts School 99-00 00-01 01-02 02-03 03-04 04-05 05-06 06-07 07-08 08-09 09-10 10-11 11-12 Year Madison Other Districts DPI 1 st Gen Metropolitan (Green Bay, (WINSS) Edvantage Edvantage Racine ) Milwaukee Milwaukee Madison Milwaukee Public Public Schools Metropolitan Finance & 1 st Gen 1 st Gen Schools Staff Edvantage DPI Edvantage
A DVANCED A NALYTICS : V ALUE - A DDED AND E DVANTAGE
V ALUE -A DDED P ARTNER D ISTRICTS
T HE P OWER OF T WO & Achievement Value-Added Compares students’ Measures students’ individual performance to a standard academic growth longitudinally A more Factors in students’ Does not factor in students’ complete background characteristics background characteristics picture of outside of the school’s control student Measures students’ Measures the impact of performance at a single learning teachers and schools on point in time academic growth Critical to students’ post - Critical to ensuring students’ secondary opportunities future academic success Adapted from materials created by Battelle for Kids
T HE O AK T REE A NALOGY To view a narrated version of this analogy go to http://varc.wceruw.org/Projects/wisconsin_statewide_examples/C hapterOne/index.htm
V ALUE -A DDED C OLOR C ODING These colors are meant to categorize results at a glance, but making responsible decisions based on Value-Added estimates may require more careful use of the data. General guidelines: Green and Blue results are areas of relative strength. Student growth is above average. Gray results are on track. In these areas, there was not enough data available to differentiate this result from average. Yellow and Red results are areas of relative weakness. Student growth is below average.
NUMBER OF VALUE-ADDED ESTIMATES STUDENTS +10 -10 -5 0 +5 (WEIGHTED) 30 Grade 4 MATH Numbers lower than 0 Numbers higher than 0 Grade 3 47.1 -8.1 represent growth that is represent growth that is lower than average. higher than average. 0 represents the district or state average growth Grade 4 Students are still 39.8 -2.5 Students are learning at for students. learning, but at a rate a rate faster than the slower than the district/state average. district/state average. +5.0 Grade 5 43.0
V ALUE -A DDED S CATTER P LOTS 100 These scatter plots are a way to represent Achievement and Value- Percent Prof/Adv (2010) 80 Added together Achievement 60 40 20 Value-Added 0 -15 -10 -5 0 +5 +10 +15 Value-Added (2010-2011)
V ALUE -A DDED S CATTER P LOTS A. Students know a lot and are 100 growing faster than predicted A B. Students are behind, but are C Percent Prof/Adv (2010) 80 growing faster than predicted C. Students know a lot, but are E 60 growing slower than predicted D. Students are behind, and are 40 growing slower than predicted B D E. Students are about average in 20 how much they know and how fast they are growing 0 -15 -10 -5 0 +5 +10 +15 Schools in your district Value-Added (2010-2011)
V ALUE A DDED D ATA I NTEGRATION Historically VA data was collected at individual districts and integrated into local DW’s . Each deployment was unique. A common data model, data set, and data visualization that could be used for all districts and the state was developed. VARC produces VA data sets periodically through the year. A standard loader application was created to automatically integrate this into the DW
D EMONSTRATION Value-Added Dashboard http://dashboard.demo.versifit.com/Dashboard/
N EXT S TEPS
Recommend
More recommend
