Marshalltown Dual Language Program Evaluation Group 1: DLP: Yes and - PowerPoint PPT Presentation

Marshalltown • Dual Language Program Evaluation • Group 1: DLP: Yes and Ever an ELL: Yes • Group 2: DLP: No and Ever an ELL: Yes • Group 3: DLP: Yes and Never an ELL • Group 4: DLP: No and Never an ELL

Guide for Reporting Rigorous Evaluations • US Department of Education • Equivalent Control Groups and Treatment Groups Guidelines • All Experimental and Control Groups across all cohort groups met the requirements for having equivalent baseline scores on the Iowa Assessments in third grade in Reading Comprehension. • Groups 1 (N=466) and 2 (N=1,370) across all 8 cohorts • Groups 3 (N= 85) and 4 (N=1,698) across all 8 cohorts

Null Hypothesis • There is no significant difference between cohorts of Group 1 students and cohorts of Group 2 students scores on the Iowa Assessment in: • Reading comprehension, • Mathematics, and • Science. • .05 level

Iowa Assessment: Reading Group 1 Group 2 Dual Language Yes Dual Language No Ever in an ELL Program Yes Ever in an ELL Program Yes Standard Standard Significant Mean NCE Effect School Year Grade N Mean NCE Deviation Mean NCE Deviation Difference Difference Size 2017-2018 4th Grade 54 34.57 16.16 36.16 15.06 No -1.59 2017-2018 5th Grade 39 46.73 16.70 43.00 18.00 No 3.73 2017-2018 6th Grade 58 40.60 16.97 43.96 18.05 No -3.36 2017-2018 7th Grade 48 48.08 19.02 48.55 20.13 No -0.47 2017-2018 8th Grade 37 46.64 17.47 44.64 18.64 No 2.00 2017-2018 9th Grade 39 53.28 13.70 51.17 15.81 No 2.11 10th 2017-2018 Grade 50 57.20 15.56 48.58 14.06 Yes 8.62 0.61 11th 2017-2018 Grade 43 43.00 10.64 43.50 13.90 No -0.50 Average difference: 1.31

Iowa Assessment: Mathematics Group 1 Group 2 Dual Language Yes Dual Language No Ever in a ELL Program Yes Ever I an ELL Program Yes Standard Standard Significant Mean NCE Effect School Year Grade N Mean NCE Deviation Mean NCE Deviation Difference Difference Size 2017-2018 4th Grade 54 35.77 15.13 37.85 15.02 No -2.08 2017-2018 5th Grade 37 34.54 15.25 36.08 18.18 No -1.54 2017-2018 6th Grade 58 36.36 16.28 42.08 13.91 Yes -5.72 0.39 2017-2018 7th Grade 48 47.79 14.28 46.47 17.38 No 1.32 2017-2018 8th Grade 37 44.70 15.74 43.16 14.74 No 1.54 2017-2018 9th Grade 42 47.11 17.36 49.80 15.67 No -2.69 10th 2017-2018 Grade 49 52.11 16.64 45.03 15.75 Yes 7.08 0.45 11th 2017-2018 Grade 38 50.18 12.40 48.28 15.04 No 1.90 Average difference: 0.02

Iowa Assessments: Science Group 1 Group 2 Dual Language Yes Dual Language No Ever in an ELL Program Yes Ever in an ELL Program Yes Standard Standard Significant Mean NCE Effect School Year Grade N Mean NCE Deviation Mean NCE Deviation Difference Difference Size 4th 2017-2018 Grade 54 40.55 14.40 42.29 12.06 No -1.74 5th 2017-2018 Grade 37 43.27 12.87 39.24 14.06 No 4.03 6th 2017-2018 Grade 58 37.25 17.24 39.08 13.86 No -1.83 7th 2017-2018 Grade 48 50.45 13.42 47.77 15.95 No 2.68 8th 2017-2018 Grade 37 51.78 18.61 50.75 13.89 No 1.03 9th 2017-2018 Grade 42 48.83 15.42 48.83 16.69 No 0.00 10th 2017-2018 Grade 50 52.78 16.23 47.32 1.90 Yes 5.46 0.42 11th 2017-2018 Grade 39 45.97 17.17 47.56 14.61 No -1.59 Average difference: 1.01

Conclusions Group 1 and Group 2 • Iowa Assessment scores in Reading: • No Clear Differences or Patterns Across Cohorts • Except 2017-2018 10 th Grade Cohort • Iowa Assessment scores in Mathematics • No Clear Differences or Patterns Across Cohorts • Except 2017-2018 10 th Grade Cohort • Iowa Assessment scores in Science: • No Clear Differences or Patterns Across Cohorts • Except 2017-2018 10 th Grade Cohort

Conclusions 2017-2018 10 th Grade Group 1 and Group 2 • Significant Differences • 2017-2018 10 th Grade (N=50) Cohort Group 1 • Reading: Significant difference • Mean NCE difference= 8.62 • Effect size = .61 • Mathematics: Significant difference • Mean NCE difference = 5.46 • Effect size = .42 • Science: Significant difference • Mean NCE difference = 7.08 • Effect size = .45

Group 1 Group 2 Dual Language Program Yes Dual Language Program No Ever in an ELL Program Yes Ever in an ELL Program YES Not Significantly Significantly School Year Grade Different Percentage Different Percentage Total 2017-2018 11th 0 0.0% 10 100.0% 10 *2017-2018 10th 7 33.3% 14 66.7% 21 2017-2018 9th 0 0.0% 8 100.0% 8 2017-2018 8th 1 14.3% 6 85.7% 7 2017-2018 7th 0 0.0% 6 100.0% 6 2017-2018 6th 1 20.0% 4 80.0% 5 2017-2018 5th 0 0.0% 4 100.0% 4 2017-2018 4th 0 0.0% 4 100.0% 4 Across all Grades & Subjects 9 13.8% 56 86.2% 65

Null Hypothesis • There is no significant difference between cohorts of Group 3 students and cohorts of Group 4 students scores on the Iowa Assessment in: • Reading comprehension, • Mathematics, and • Science. • .05 level

Iowa Assessment: Reading Group 3 Group 4 Dual Language Yes Dual Language No Never an ELL Never an ELL Standard Standard Significant Mean NCE School Year Grade N Mean NCE Deviation Mean NCE Deviation Difference Difference Effect Size 2017-2018 4th Grade 16 45.93 24.39 45.75 23.92 No 0.18 2017-2018 5th Grade 6 55.00 31.36 38.66 25.08 No 16.34 2017-2018 6th Grade 7 58.42 22.75 51.85 19.59 No 6.57 2017-2018 7th Grade 9 64.77 24.65 48.33 14.97 No 16.44 2017-2018 8th Grade 6 49.55 13.01 47.00 11.15 No 2.55 2017-2018 9th Grade 12 61.75 22.65 50.66 17.71 No 11.09 10th 2017-2018 Grade 5 61.80 13.66 54.60 16.19 No 7.20 11th 2017-2018 Grade 4 74.25 16.64 60.25 14.33 No 14.00 Average difference: 9.3

Iowa Assessment: Mathematics Group 3 Group 4 Dual Language Yes Dual Language No Never an ELL Never an ELL Standard Mean Standard Significant Mean NCE School Year Grade N Mean NCE Deviation NCE Deviation Difference Difference Effect Size 2017-2018 4th Grade 16 42.62 21.30 34.68 15.64 No 7.94 2017-2018 5th Grade 6 44.00 18.18 42.33 26.84 No 1.67 2017-2018 6th Grade 7 58.85 19.31 48.57 15.59 No 10.28 2017-2018 7th Grade 9 55.66 17.08 48.33 16.19 Yes 7.33 0.45 2017-2018 8th Grade 6 46.67 11.25 49.00 13.76 No -2.33 2017-2018 9th Grade 12 53.00 17.49 50.41 18.03 No 2.59 2017-2018 10th Grade 5 55.40 27.57 58.60 10.25 No -3.20 2017-2018 11th Grade 4 73.00 11.28 65.75 17.63 No 7.25 Average difference: 3.94

Iowa Assessment: Science Experimental Group 3 Control Group 4 Dual Language Yes Dual Language No Never an ELL Never an ELL School Standard Standard Significant Mean NCE Year Grade N Mean NCE Deviation Mean NCE Deviation Difference Difference Effect Size 2017-2018 4th Grade 16 52.18 18.03 45.00 24.62 No 7.18 2017-2018 5th Grade 6 54.00 16.52 44.00 16.66 No 10.00 2017-2018 6th Grade 7 48.57 15.39 51.28 14.27 No -2.71 2017-2018 7th Grade 8 57.00 22.23 50.87 11.38 No 6.13 2017-2018 8th Grade 6 49.17 12.41 61.50 16.48 No -12.33 2017-2018 9th Grade 12 57.00 22.63 46.91 16.90 No 10.09 2017-2018 10th Grade 5 60.00 9.94 57.20 9.98 No 2.80 2017-2018 11th Grade 4 74.25 16.64 60.25 14.33 No 14.00 Average difference: 4.4

Group 3: DLP Yes, Never an ELL Group 4: DLP No, Never an ELL • Iowa Assessment scores in Reading: • No Significant Differences Across the 8 Cohorts • Small N Sizes • When combining all cohorts together there is a significant difference favoring DLP • Iowa Assessment scores in Mathematics • Seven No Significant Differences Across the 8 Cohorts • One Significant Difference across the 8 Cohorts favoring DLP • Small N Sizes • When combining all cohorts together there is a significant difference favoring DLP • Iowa Assessment scores in Science: • No Significant Differences Across the 8 Cohorts • Small N Sizes • When combining all cohorts together there is no significance difference

Group 3 Group 4 Dual Language Yes Dual Language No Never an ELL Never an ELL School Mean Standard Mean Standard Significant Mean NCE Effect Year Subjects Grades N NCE Deviation NCE Devaition Difference Sig. Difference Size 2017- 2018 Reading 4th - 6th 28 52.64 23.7 46.53 22.91 No 0.109 6.11 2017- 2018 Math 4th - 6th 28 48.03 20.16 41.03 17.32 No 0.091 7 2017- 2018 Science 4th - 6th 28 52.64 16.13 47.96 18.97 No 0.247 4.68 2017- 7th - 2018 Reading 11th 36 60.6 19.67 50.22 14.44 Yes 0.001 10.38 0.72 2017- 7th - 2018 Math 11th 36 55.16 18.14 52.5 16.16 No 0.279 2.66 2017- 7th - 2018 Science 11th 35 58.05 19.29 53.31 14.97 No 0.186 4.74 Average difference: 5.92

Conclusions • Combining Cohorts across Groups 3 (DLP but Never and ELL ) and Combining Cohorts across Groups 4 (Not in the DLP and Never an ELL) • There were two significant differences favoring students in the Dual Language Program, one in reading and one in mathematics • In the 30 significance tests ran two (7%) revealed a significant difference between groups 3 and 4, favoring Group 3 while (93%) revealed no significant difference between groups in reading, math and science scores on the Iowa Assessments.