Unpacking the Learner- Selection Suitcase : A Synthesis of Evaluation Findings from Learner – Directed Educational Improvement Initiatives. Benita Williams Mobile: +27 82 772 9709 bwilliams@feedbackpm.com
“Case” Study Questions 1) You have to select 100 learners for a two year FET Saturday school programme running in the district. A third of the “top learners” score below 30% on a standardized assessment of Grade 10 Maths knowledge. Do you select them? 2) You want to select learners for a post-matric programme. You have WAIS III scores, scholastic test scores in Maths and English, past matric results, and face to face interview notes, How do you combine all of these to make a recommendation? Do you think all of these tests are necessary? 3) You want to select learners with the best chance of success in an enrichment programme. Do you rather select a learner with past performance of 30% or 60%?
Three “Case” Studies… Case 1: Supplementary ICT Education Initiative – Using School Assessment Results to Select Learners
Case 1: Supplementary ICT Education Initiative Target - “learners with potential” Activities 1) Mathematics Saturday classes, 2) Mid-week sessions using ICT technology to do project work intended to enrich their understanding of the Mathematics and Physical Science content taught in class, and 3) mid-week sessions for learners engaging in self-paced learning with online content intended to further learners’ understanding of basic Mathematics and Physical Science concepts
Case 1: Supplementary ICT Education Initiative Objectives Develop independent, self driven, self motivated learners, with good quality Mathematics and Physical Science passes. Learners who emerge from the programme would be inspired to pursue careers in the Mathematics and Science fields, Learners will be emotionally and academically prepared to enter tertiary institutions. Selection Tools Districts Select Schools from townships that are functional and have access to ICT labs Schools select “top” learners based on school results – must have maths and science and pass both learning areas Exclude learners involved in other projects
Case 1: Supplementary ICT Education Initiative Selection Result Selected the learners, and tested them independently at the Saturday school almost all got 25% - 35% Facilitators of ICT skills classes reported that the learners able to learn the ICT skills, but that they are worried that the programme content are pitched at learners who are supposed to have a basic understanding of maths and science concepts Some of the learners could not do the self paced learning in the ICT lab, because their school did not have a lab Some of the learners dropped in and out – depending on their participation from other schools
Range of Individual Learner's T otal T est Marks 100 90 80 70 Learner Marks Learners With Potential- Strict Definition 60 50 Learners With Potential- Expanded Definition 40 30 20 10 0 0 1 2 3 4 5 6 7 8 9 School A School B School C School D School E School F School G School H Boitumelong Bokomoso Eketsang Katlehong Masisebenze Mpilisweni Tembisa
Case 1: Lessons Learnt Drawing learners from different schools into a programme, requires a comparable measure across schools. If the programme is targeted at a specific performance level, the measurement needs to exclude those who do not qualify Practical constraints have a bigger impact on project success than previous performance ( Participation a big issues.
Case 2: Saturday School Programme for 400 learners in four non-urban sites – Using Standardised Maths and Science tests to Select Learners
Case 2: Saturday School Programme Target: “Bright Minds” from rural schools in a 60km radius from a Saturday School facility Learners with 40% - 60% in Grade 10 Activities: T wo years of Saturday School and holiday tuition in Maths Physical Science English Career Guidance Life Skills Computer Technology Training
Case 2: Saturday School Programme Objectives Increase the number of good quality passes in maths and science Open pipeline for entrance to further studies in SET careers Selection Tools T ested 700 learners from about 76 schools identified by the DBE Used a standardised test administered by an external service provider Selected “top” 100 learners from each district
Mathematics Results for the Control and Experimental Groups Control Experimental Linear (Control) Linear (Experimental) 100 90 y = 0.8624x + 32.259 R² = 0.1045 MatricDistinctions 80 70 Mathematics Matric Exam Mark 60 y = 1.1831x + 13.14 R² = 0.1377 50 40 30 20 10 0 0 10 20 30 40 50 60 70 80 90 100 CutOff = 22 % Mathematics Selection Test Mark
Case 2: Saturday School Programme Distinction Candidates - Selection test and Matric results Selection Test Matric Exam 100 80 Test Result 60 40 20 0 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55
40.0 30.0 Change in mark from pre to post-test 20.0 Maths Physical Science 10.0 English Linear (Maths) Linear (Physical Science) Linear (English) 0.0 0 to 70% 71% to 76% to 81% to 85% to 90% to 96% to 100% 75% 80% 85% 90% 95% 99% -10.0 -20.0 Attendance Rate
Case 2: Lessons Learnt The selected participants weren’t what they looked for, but the programme could be adapted - Good idea! If the donor kept to the original criteria, a number of learners who was able to produce good quality passes, would have been excluded Still, a number of learners who could have produced good quality passes were excluded Misclassification is more likely if you use only one test result.
Case 3: Post-matric programme T o help learners improve their maths results – Using a battery of psychological and scholastic tests.
Case 3: Post-matric programme Target: Learners under the age of 21 years Have completed the Grade 12 year of schooling Have previously sat the matric examinations Have mathematics, and Physical Science and/or accounting as Grade 12 school subjects Be able to attend the Programme full time for one year Be able to obtain a Grade 12 matriculation by the end of the programme
Case 3: Post-matric programme Activities: 8 months of tuition 08:00 to 17:00 Monday to Friday Maths & Physical Science / Accounting IEB exam preparation Career Guidance, Life Skills Computer T echnology English for non-exam purposes Assistance with bursary applications, study applications, job applications Objectives Improve number of good quality passes Emotionally prepared for further studies and world of work
Case 3: Post-matric programme Selection Tools Initial Application form “Mental Alertness” test Assessment Centre WAIS III Clinical Observations Thematic Apperception Test & Draw a Person Test Grade 12 Reading Comprehension Test (old Grade 12 exam paper) A spelling test (Called the “Spelling Test for Psychologists A Mathematical Test developed by a school teacher to be representative of a typical Grade 12 Mathematics Assessment. Interactive Group Assessment – Panel Selection – March Exams: Final Selection
Case 3: Post-matric programme Selection Result Not necessarily above average in terms of aptitude Excludes learners who are decidedly below average ito aptitude. Not very well prepared scholastically – even learners who have relatively higher aptitude, perform poorly on the scholastic tests. Do not have insurmountable emotional challenges, Able to express themselves at least at a very basic level in English Likely to come from impoverished backgrounds, but at least able to function socially and occupationally Able to negotiate their social environments. Those who are unable to adapt to the programme demands, or able to reconcile their home circumstances with the performance drops out
2009 Cohort Pre and Post Maths Marks Pre Post 100% 80% 60% 40% 20% 0% 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 Average difference between the pre and post maths marks was 9.9%. Best predictor of improvement – previous performance. The weaker, the bigger improvement
Points to ponder Why Select? Is selection necessary – What are the Programme pitch or financial considerations? Even if the donor can afford to “select” everyone, remember there are costs of participation for beneficiaries Are you selecting “In” or screening “out” What counts more – past performance or current participation? You may select, but the participants still get to elect to participate or not.
Points to ponder Select with which tools? Choose a selection strategy, test it, and refine it based on the programme requirements. Select Whom? Learners with potential “T op learners” “Bright Minds”
Recommend
More recommend
