Learning Science: The Importance of Intervention Research Barbara Schneider and Joseph Krajcik Relevant Research Roundtable-R3 Florida State University March 9, 2020
Outlin line of e of t the P e Pres esentatio ion • Why We Need Evidence for Science Policy and Practice . Why Interventions are Critical Principles and Criteria for Testing an Intervention effect • How to Form Evidence on Social and Emotional Learning • What the First Steps in Developing a Theory of Action— Example from Crafting Engagement in Science Environment • Why Develop a Logic Model • Procedures, Measures, and Results • What We are Learning
Why We Ne Need ed I Interven ention ons • Worldwide policymakers have encouraged the use of scientifically based evidence to make decisions for supporting specific educational programs and practices. • To be able to do this information is needed about what programs and practices do or do not work. • To produce such evidence leads to causal questions, such as whether particular programs and practices improve student academic achievement, social development, and pathways to further education, earnings, and civic participation.
Principles of Interventions As scientists should adopt principles to assess the trustworthiness of evidence by making: 1) Claims supported by evidence 2) Identify fair comparisons 3) Goals, problems & designs defined 4) Produce reproductible & replication effects
Criteri ria f for T r Testing an Intervention Effect 1) Intervention based on theoretically grounded principles 2) Assumptions about what the impact will be (including pre-registration of the study--SREE) 3) Possible to achieve a measurable effect (Power Effect—Optimal Design Software) 4) Sound measures of face and content validity (standards –performance expectations NGSS and rubrics 5) Reliable measures and tools (assessments—NAEP and MDE) for making claims about the intervention effects 6) Rigorous Analytics including sensitivity analysis and examination of heterogeneity differences
Our C Challenge Build learning environments that: • Foster deep and integrated understanding of important idea • Engage students, i.e., create optimal learning environments, in learning science • Support students in developing important scientific practices and 21 st century competencies • Support students to solve problems, think critically, and innovatively
Crafting En Engagement in Science En Envi vironments: The T e Trea eatmen ent • Goal, develop and test a system for advancing science teaching and learning that builds a vision for enacting project-based learning and meeting NGSS for High School Chemistry and Physics. • The system includes: • Highly developed and specified educative teacher materials • Highly developed and specified student materials • Professional learning supports • 3-dimensional formative and end-of-unit assessments
Crafting E Eng ngag aging Science ce Environments (CESE) NSF funded, Multi-year project ● International Collaboration: US and Finland ● Large scale study - 130 teachers, 70 schools, ~8000 students ● Goal: ● “ To increase student engagement and interest in the fields of science, technology, engineering, and mathematics (STEM)”
Impor ortance o ce of Building A An Inter erdisci ciplinary T Team You cannot know everything ● Studying an Intervention requires substantive knowledge of science learning --Project Based Learning—Joseph Krajcik Expertise in content knowledge of the intervention Solid analytic expertise— Building an interdisciplinary team of Graduate students, Undergraduates, Post Doctoral Fellows, Content Specialists Willingness to share, negotiate, transparent, and open about challenges and disappointments.
Our Claims Our design process allows us to create PE-aligned units and ● assessment tasks Students learning these units show growth in their ability to answer ● three-dimensional assessment items Students show an improvement in their engagement, OLM and ● other socio-emotional constructs
Theory of Engagement for Science Learning
Logic Model
Putting it all together • How do we create units that Project Based Learning aligned with the new vision for science education? • How do we create three- dimensional assessment items? New Vision for • Do our units allow for the Science development of skills and Education knowledge alongside Optimal Learning Moments Curriculum & students engagement? Assessment
Our Units Unit Name Driving Question Phenomena Performance Expectations Forces and Motion “How can I design a vehicle to be safer car\cart collision HS-PS2-1 for a passenger during a collision?” HS-PS2-3 MagLev “How do mag-lev trains function Magnetic Levitation HS-PS3-5 without touching the track?” HS-PS3-2 Electric Motor “How can I make the most efficient Toy motors HS-PS3-1 electric motor?” HS-PS2-5 HS-PS3-3 Evaporation “When I am sitting by the pool, why Evaporation of different HS-PS1-3 do I feel colder when I am wet than liquid on the palm of your HS-PS3-2 when I am dry?” hand Periodic Table “Why is table salt safe to eat but the Reaction of Sodium with HS-PS1-1 substances that forms it are explosive water HS-PS1-2 or toxic when separated?” Conservation of “Can I make substances appear or Flash paper, invisible ink, HS-PS1-7 Mass disappear?” Al/CuCl 2 reaction
Measuri ring Social and Em Emotional Lea earnin ing Value of of R Rep epli licatio ion of of Sc Scales PIRE When working on this activity…I used my imagination. When working on this activity…I solved problems that had more than one possible solution. When working on this activity…I explored different points of view on the problem or topic. When working on this activity…I had to make connections with other school subjects. OECD When working on this course...I have to use my imagination. When working on this course...I have to solve problems that have more than one possible solution. When working on this course...I have to explore different points of view on a problem or topic. When working on this course...I have to make connections with other school subjects.
Experience Sampling Method (ESM) • Focus on the situational and contextual aspects of what happens in and out of the classroom • Short, repeated surveys capture what students are doing and feeling in-the-moment ( Csikszentmihalyi, 1975) • Less opportunity for recall bias and socially desirable answers (Hektener et al., 2007) • Students signaled randomly during the course of a week or days.
Measuri ring Social and Em Emotional Lea earnin ing Value of of R Rep epli licatio ion of of Sc Scales PIRE When working on this activity…I used my imagination. When working on this activity…I solved problems that had more than one possible solution. When working on this activity…I explored different points of view on the problem or topic. When working on this activity…I had to make connections with other school subjects. OECD When working on this course...I have to use my imagination. When working on this course...I have to solve problems that have more than one possible solution. When working on this course...I have to explore different points of view on a problem or topic. When working on this course...I have to make connections with other school subjects.
Figure 1. Field Test Single Case Design Graph 18
New R ew Results • Importance of Measuring Relationship of Social and Emotional Measures for both treatment and Control Conditions-
New R ew Results • Importance of Measuring Relationship of Social and Emotional Measures for both treatment and Control Conditions-
Treatment students feel less successful and confident during low-challenge moments but more successful and confident during high-challenge moments 1545684
A one e standard d dev eviation increa ease i in aggr greg egated ed situational engagemen ent c t corres esponds to to a 0.15 s standard d dev eviati tion in increase in in summative c e course g e grade. e.
Implications Treatment classrooms focused on asking questions about phenomena and exploring a relevant driving question An emphasis on scientific practices, while still challenging, offered students opportunities to demonstrate their competence and feel successful in science Project-based learning may take teachers and students out of their comfort zones, but can lead to payoffs in science learning and social and emotional experiences 1545684
What h has happened? • A large main effect on science achievement for the intervention; most valuable for low-income and minority students • Raised imagination and desire to take on challenging problems to figure things out • Increases in interest in pursuing science courses in college and later careers • Growth in teacher engagement with scientific practices
Networking Our professional learning workshops provide an opportunity for teachers across the country to connect and discuss teaching methods, share resources, student testimonials, lesson modifications, and more.
Recommend
More recommend
