Handbook For Professional Development In Assessment Literacy: A - PowerPoint PPT Presentation

Handbook For Professional Development In Assessment Literacy: A Resource For States and School Districts NCSA Conference June 20, 2013

REVISION OF THE HANDBOOK FOR PROFESSIONAL DEVELOPMENT IN ASSESSMENT LITERACY Project of the Title I Comprehensive Assessment Systems CCSSO SCASS Wayne Neuburger (PhD) Advisor

T1-CAS Mission • The Title I Comprehensive Assessment Systems SCASS supports states in efforts to use assessment and accountability systems to support the improvement of education in schools using ESEA funds. • As a national consortium of assessment and Title I professionals, T1-CAS addresses issues in standards, assessment, and accountability systems, and the effects of these systems on the education of Title I students.

Presenters • Moderator: Wayne Neuburger, Consultant • Project Director: Jan Sheinker, Consultant • Author: Doris Redfield, Consultant • Reviewer: Beth Cipoletti, West Virginia Dept of Ed • Discussant: Elizabeth Davis, Alaska Dept of Ed

Jan Sheinker, Ed.D. Doris Redfield, Ph.D. Developed Under the Direction of: Phoebe C. Winter Project Director And The Professional Development for Assessment Literacy Study Group Comprehensive Assessment Systems for IASA Title I State Collaborative on Assessment and Student Standards Council of Chief State School Officers

Purpose 2001 and 2013 • It is intended to provide a resource to states and districts as they deploy state and district assessment systems aligned with standards for purposes of improving student learning through accountability and school improvement . • We hope that this document also serves as a resource for informing many constituents of education about the purposes of assessment and the importance of an aligned assessment system to the overall educational system .

SCASS GROUPS PROVIDING INPUT • Formative Assessment for Students and Teachers (FAST) • Assessing Special Education Students (ASES) • English Language Learners (ELL) • Accountability Systems and Reporting (ASR) • Technical Issues in Large-Scale Assessment (TILSA)

Need for Revision • Significant changes have occurred in assessment systems since 2001. • Accountability systems have changed since 2001. • Assessment technical issues have progressed to address new assessment and accountability systems. • Distribution systems are more sophisticated

Organization and Content of the Handbook Jan Sheinker jansheinker@gmail.com

Organization – Actually FOUR documents Documents Users Uses States, districts, schools, and To preserve the original document PDF other interested audiences State and district PD For customization by individual Word presenters states, districts, & schools as PD scripts and handouts for PD And presenters Individual states, districts, & For customization by individual Ppt schools states, districts, & schools as district (Static & and school newsletter inserts to Animated) inform parents and community

PowerPoint for use in Trainings & Presentations

Table of Contents Guide to Using the Handbook Chapter One: Why Build an Assessment System? Chapter Two: What Is Technical Quality? Chapter Three: How Are the Purposes of Assessment Related to Technical Quality? Chapter Four: How Are the Uses of Assessment Related to Technical Quality? Chapter Five: How Do Schools/States Report Results in Proper Context? Chapter Six: How Should Results Be Used to Make Decisions? Appendices Glossary References

Guide to Using the Handbook Using the Handbook Audiences for the Handbook State Department Personnel Schoolwide Planning Participants Local School Boards and Administrators Legislative Committees Regional Professional Development Participants Members of Professional Associations Pre-Service and Graduate Students in Education Community Members Customizing C ONCLUSION

Chapter One: Why Build an Assessment System? • Standards Aligned Assessment Systems – How have aligned assessment systems evolved? – How have Common Core State Standards influenced the development of assessment systems? • Relationships of Tests to Assessment Systems to Accountability Systems – What are comprehensive assessment systems? – What are the purposes and relationships among formative assessment strategies and classroom, school, district, and state tests? – How do assessment systems relate to accountability systems? – Why use an assessment system instead of individual tests to set goals and make decisions? • Developing a Comprehensive System – What is the role of formative assessment strategies in a comprehensive system? – What is the role of classroom tests in a comprehensive system? – What is the role of interim and benchmark tests in a comprehensive system? – What is the role of district tests in a comprehensive system? – What is the role of English language proficiency tests in a comprehensive system? – What is the role of state tests in a comprehensive system? • Assessment Systems and the Classroom – How do assessment and accountability systems relate to the way we do business in classrooms? – Who does what in standards-based schools?

Chapter Two: What is Technical Quality? • Aspects of Technical Quality • Validity – Why is validity important? – How can validity be increased? • Reliability • Fairness, Bias and Accessibility • Comparability • Procedures for test administration, scoring, data analysis, and reporting • Evaluation of the technical quality of accommodations

Chapter Three: How are the Purposes of Assessment Related to Technical Quality? • Alignment of Assessments with Standards – What is alignment with the purpose of the assessment? – Why are specific characteristics important to alignment? – Why are vertical and horizontal alignment important? • Accountability Purposes – Why is accountability needed? – Who is accountable for student learning? – For what should schools be held accountable? – What is accountability for growth? • Assessments Purposes – Why do assessment systems include different types or combinations of tests? – What are norm-referenced tests? – What are standards-based tests? – What are augmented assessment systems? – What are computer adaptive tests? – What are APIP enabled technology enhanced assessment systems?

Chapter Four: How are the Uses of Assessment Results Related to Technical Quality? • Using Results Appropriately to Avoid Misuses – What are appropriate uses and potential misuses of standards-based tests? – What are appropriate uses and potential misuses of norm-referenced tests? – What are appropriate uses and potential misuses of interim/benchmark tests? – What are appropriate uses and potential misuses of classroom tests? – What are appropriate uses and potential misuses of formative assessment strategies? • Usability of Results – What should be considered in determining the usability of results? – What factors affect the credibility of results? – What factors affect the accuracy of score interpretation? – What results are needed to adjust instruction? – What factors affect the usefulness of results for teacher and leader evaluation?

Chapter Five: How Do Schools/States Report Results in Proper Context?  Reporting Results - How are indicators selected for reporting? What indicators provide direct measures of student achievement of - standards? What other indicators provide direct measures of student knowledge? - What student learning indicators provide indirect measures of student - performance?  Reporting Related Indicators What indicators provide measures of opportunity to learn? - What context variables affect student learning? -  Cautions for Reporting Results  Sampling and Sample Size How does sampling affect the technical quality of a tests? - How does population and sample size affect the technical quality of a test? -

Chapter Six: How Should Results Be Used to Make Decisions? • Using Results to Make Decisions • Using Results to Make Decisions About School Improvement – How can the results be used to profile student performance? – How is the profile used to set school improvement goals? – What is considered in developing a plan to achieve the goals? – How are school improvement results monitored and documented? • Using Results to Make Decisions about Policy Changes and Evaluating program effectiveness – How can the system be monitored and evaluated? – How can results be used to make policy decisions concerning resources? – How can the delivery system be altered to improve results? • Using Results to Make Decisions About Rewards and Sanctions

Technical Quality (Chapter 2: Handbook for Professional Development in Assessment Literacy) Doris Redfield, Ph.D. dredfield@suddenlink.net 304-344-3083

Contents • Aspects of Technical Quality • Validity – Importance – Ways to increase • Reliability • Fairness, Bias, & Accessibility • Comparability • Procedures: Test Administration, Scoring, Data Analysis, Reporting • Evaluation of Accommodations Use

What is Technical Quality (TQ) • The integrity of each assessment, instrument, process, & procedure for contributing to fair and defensible decisions.