3D Science in the Oklahoma School Testing Program (OSTP) National Conference on Student Assessment June 2018
The Journey So Far
2014 New Standards • Frameworks Informed • Stakeholders and a Strategic Process • Now Implementation … moving from KPCOFGS to, for • example, developing a model that can be used to describe how matter is conserved during unobservable chemical processes. Nope Social media networks – OKSci • Consortiums and OSTA • PD on your Plan • Face to Face 3D PD • District MSP collaborations and shared • resources Oklahoma Frameworks (volunteers at first) • Assessments •
2015 8 th Grade Assessment Item Properties of Matter - Prior to new assessments
2016 8 th Grade Assessment Item on Properties of Matter - New 3D Standards
A person pedaled to turn Gear A, which caused the chain attached to the gear to move. This, in turn, caused Gear B and the back wheel to spin, producing electricity and charging the battery. The students noticed that Gears A and B turned at different rates. The students learned this difference in rate is called gear ratio. The third diagram shows how gear size affects gear ratio. Gear speed is measured in rpm (revolutions per minute).
What is assessed? How is it assessed? How will alignment be evaluated?
Traditional Alignment and Three Dimensions
Grade 5 8 of the 22 clusters evaluated by panelist were found to be at the DOK level of the Performance Expectation.
• Grade 8: 5 of the 22 were found to be aligned • Grade 11: 10 of the 22 were found to be aligned What went wrong? - Only used the PE - Panelist experience - DOK and three dimensions - A range of complexity is needed for IRT - Item Response Theory - Test construction/build - Accurate reporting
Why is DOK Problematic? • It has worked consistently and reliably in the past. • Still viable for other standards • But three dimensional standards have junk in the trunk
Appendices
What is the Solution? • 2017 Item Review Committee for 11th grade CCRA Science • Developed Oklahoma-specific alignment tool to evaluate item alignment and cognitive complexity compared to the standard. • Test and Item Specifications were revised to exclude DOK as an alignment indicator • 2018 Item Review for grades 5, 8, and 11 • Revisions were made and it will be used for all science • DOK will be removed from TIS and Blueprints
DOK requirements • are removed Reporting is limited • compared to previous years Overall Score • Life Science • Domain Physical Science • Domain
Alignment and Cognitive Complexity Tool • Stimulus: phenomena based, necessary for sense making • SEP: at appropriate level and grade band • CCC: implicit or explicit • DCI: Grade appropriate, difficulty level • Overall, does the cluster represent the intent of the standard? • Transfer
Peer Review……… • Evidence to support three dimensionality was asked for • Asked for empirical evidence of each dimension (as a reporting category) to support claims of three dimensionality • Performance Level Descriptors • Test and Item Specifications • Item Review Committees • Item bias and sensitivity • Standard Setting • Suggested standards be excluded as not assessable. • Traditionally, DOK 4 items are not assessable with multiple choice items • Suggestions to improve alignment (based on alignment study)
Cora James Oklahoma State Department of Education 405-522-0283 Cora.James@sde.ok.gov
Recommend
More recommend
