U.S. National Assessment of Educational Progress (NAEP) Virtual Science Lab Lei Liu Shu-Kang Chen Timothy Fiser Educational Testing Service William Tirre National Center for Education Statistics Presentation at NCSA 2015 San Diego, CA
Acknowledgement Development team Research team • Shu-Kang Chen • Lei Liu • Timothy Fiser • Helena Jia • Raymond De Hont • Katherine Castellano • Elia Mavronikolas Ex-external developer: • Sarah Ohls eLearning Brothers, Inc. Infuse, Inc. NAEP SAIL Executive Directors • Joanna Gorin • Gregory Vafis • Andreas Oranje
Overview • What drives this project • Project objective • Design principles • The student model • Three-phase plan and update • What we have learned • Next Steps
Innovation in Science Assessment • Future science assessment must • improve measurement of “hard -to- measure” science constructs that are core of NGSS (e.g., science practices), and • increase the feasibility of measuring additional constructs including critical thinking, problem solving, metacognition, and other 21 st Century skills.
Project Objective • To develop a tablet-based virtual science laboratory (VSL) to measure students’ complex scientific competence and provide evidence about how well they can apply content knowledge and science practices through different paths of problem solving.
VSL Design Principles Traditional Approach Innovative Approach Follow a “ cookbook ” approach of Follow inquiry-based approach by replicating or verifying certain scientific providing free space for students to phenomena inquire knowledge (e.g., trying out experiments) Gather and record data without a clear Problem-based knowledge inquiry by sense of the purposes of their using lab equipments to design investigation experiments Collect data that are predefined Conduct real experiments to collect scientifically accurate data that are not predefined in any way Experiments are dependent on Develop modular activities to support individual science topics thus are costly reusable capability and to possibly reduce the cost of development
The Student Model Science Practices -Planning investigations; Inquiry Science Practices -Carrying out investigations; Skills -Analyzing and interpreting data; -Communicating NAEP Crosscutting NGSS Concepts Science Framework Science Disciplinary Core Content Ideas Core Ideas -Matter and its interactions -Forces and energy -Waves and energy
VSL Development Plan Modular Feasibility Prototype Activities Phase: Phase: Phase: A proof-of- A prototype Prototypes concept of the of four prototype to overall modular determine setting and activities in the a chemistry the technical lab chemistry needs lab
Feasibility Demo
Workstation • Mixing two liquids • The simulated chemical reaction (e.g., color change) responds to the direct input of user in real time
Prototype Phase – Six Action Areas
Action Area 1: Virtual Laboratory • Three labs for students to explore (as a long-term goal) • A 3D virtual chemistry lab • Have access to all other action areas through touch in the lab or the GUI dashboard
Action Area 2: Lab Supply • View and select interactive objects for lab use • View detail info about each object • Navigate 3D objects • Drag and drop any object from the shelves to a lab cart
Action Area 3: Lab Cart • Add / remove selected lab supply objects to the lab cart • Indicate the quantity of selected objects • See detailed label for each selected object
Action Area 4: Workstation • Manipulate selected objects to conduct virtual experiments • Interact with 3D objects in relative scale to try out relevant features • Change view of objects by zooming, panning, and/or rotating the viewing position • View simulated results when interacting with 3D objects
Action Area 5: Notepad • Record and refine design and results of experiments • Identify relevant variables • List relevant lab supplies • Plan experimental procedures • Record data • Identify qualitative/quantitative data patterns • Draw conclusions • Respond to direct assessment items embedded in the VSL
Action Area 6: Whiteboard • Share products generated in the Notepad • Communicate design decisions • Communicate observations (e.g., data patterns) • Communicate conclusions
Modular Activities Phase
Mini Modular Lab Activities • Modular Activity 1 – Planning Experiments • selecting relevant tools • designing lab procedures • Modular Activity 2 – Carrying Out Experiments • measuring and recording weight • measuring and recording volume • Modular Activity 3 – Analyzing & Interpreting Data • using notebook to record and interpret data patterns • Modular Activity 4 – Communicating • using whiteboard / notebook to share data patterns or lab results
What has been learned? • Both the NAEP Science framework and the NGSS call for measures of complex scientific competencies • The balance between user freedom and design constraints to limit cost and timelines is difficult to achieve • Feasibility study for proof-of-concept is useful to help determine technical demands and vendor capability • Developing modular lab activities will provide several key benefits: • reducing construct-irrelevant noises and focus on constructs that the virtual lab affords to measure; • improving the reusability of the lab environment; • reducing the cost of development
Next Steps • Usability study on the design of effective user interfaces (Prototype Development Stage): • Intuitive, easy-to-learn interface and navigation • No distraction from test content • UDC considerations • Cog lab study on the construct-relevant issues (Prototype Development Stage): • What evidence can the VSL afford students to demonstrate their integrated science learning? • Can the VSL generate evidence to differentiate students’ integrated knowledge?
Recommend
More recommend
