Nepal (Using Contextual Data in National Assessment) Devi Ram - PowerPoint PPT Presentation

10-13 September, 2018, Penang, Malaysia Capacity development workshop “Conceptualization, Measurement and Use of Contextual Data” Nepal (Using Contextual Data in National Assessment) Devi Ram Acharya Education Review Office Nepal

Evolving Practice of Assessment in Nepal • Various assessments of student achievement (Grade 3, 4, 5, 8 & 10) carried out from 1995 to 2010. • Before 2010 AD, there was no responsible institution, no fixed time interval and defined sampling. • BQ questionnaire were not used. • After establishment of ERO (2010 AD), regular National Assessment practice is taking place.

Completed Assessment Year Grade Subject 2011 8 Nepali, Mathematics & Social Study 3 Nepali & Mathematics 2012 5 Nepali, English & Mathematics 2013 8 Nepali, Mathematics & Science 3 Nepali & Mathematics 2015 5 Nepali, English & Mathematics

Ongoing & Planned Assessment Year Grade Subject 2017 8 Nepali, Mathematics & Science 2018 5 Nepali & Mathematics 2019 10 Nepali, English, Mathematics & Science 2020 8 Nepali, Mathematics & Science 2021 5 Nepali & Mathematics 2022 10 Nepali, English, Mathematics & Science Integrated (Literacy & Numeracy) assessment has been planned for Grade 3

Glimpse of Assessment Name of the assessment National Assessment of programme: Student Achievement Scope National Coverage Sample Grade level: primary (e.g. Primary & Secondary grade 1, 2 ) or secondary (Grade 5 and 8) Domains (e.g. reading, Language (Reading & mathematics) Writing), Mathematics & Science Contextual/background Student, Teacher & HT questionnaire Analysis Descriptive & Correlational.

Sampling • Stratified random sampling (16 strata: 5 development region* 3 eco-zone plus valley) Now onward, development regions are changed in provinces. • Plan to move PPS. • School Type : Private & Public, Rural & Urban • Student: Target students (14 to 16 thousand) are randomly selected from school as proportionate.

Assessment Instrument • Student test items - 3 booklets with some anchor items • Background questionnaires – Teacher (demographic, professional and attitudinal) – Student (demographic, SES & attitudinal) – HT (demographic, professional and school management)

Framework

Issues regarding BQ • Develop the contextual questionnaire (research committee members opinion). • Fill up the questionnaire (difficult to understand by grade 3&5 student) • Some inconsistent data has found (> Master Degree) • Public understanding found different than the result. (time to spent watching TV)

Addressing the Issues • BQ questionnaire fill up guideline • Teacher support to students for fill up questionnaire. • Data clean (recode, omit) • Update & edit in next year.

Example Result (Mathematics) 16 years (N=1770) 480 15 years (N=3283) 493 14 years (N=5266) 506 13 years (N=3762) 506 12 years or belw 504 (N=783) 300 350 400 450 500 550 Achievement Scale Score Students of the age 13 and 14 years performed better than other age group.

Score by father’s education in Science Master&above 537 (N=475) Bachelor (N=690) 532 Grade 12 (N=1771) 516 Grade 10 (N=4471) 503 Literate Only (N=5462) 497 Illiterate (N=2106) 480 300 350 400 450 500 550 600 Achievement Scale Score

Performance by Mother’s Education in Nepali Master&above 515 (N=145) Bachelor (N=309) 526 519 Grade 12 (N=725) 510 Grade 10 (N=2539) 506 Literate Only (N=6100) 489 Illiterate (N=5009) 300 350 400 450 500 550 600 Achievement Scale Score

Involvement in household chores in Maths More than four hour 479 (N=837) two to four hour 492 (N=1929) one to two hour 502 (N=4892) Less than 1 hour 509 (N=4967) Don't give tome 504 (N=1527) 300 350 400 450 500 550 600 Achievement Scale Score Students who spend few hours in house hold chores are better in performance.

No leisure class 510 (N=3452) Playing (N=1274) 486 Performance by Group work Activity in Leisure 507 (N=2216) Time at School classwork/home 500 work (N=7901) 300 350 400 450 500 550 Achievement Scale Score

Time spending in Watching TV in Maths more than four hour 489 (N=98) two-four hour (N=220) 505 one-two hour (N=2065) 516 less than an hour (N=8246) 503 not given time (N=3948) 489 300 350 400 450 500 550 600 Achievement Scale Score Students who watch TV for 1 to 2 hours daily can perform better. This may be the case access to the information. To fulfil the gap between the students who have access in TV and don’t have access can be minimized by the availability and use of ICT tools in the School.

SES and Schools' Mean Score in Mathematics

Guiding questions Please pick one learning programme implemented in your country (preferably, the recent one) and respond to the following questions: • Q.1 Briefly describe the assessment programme by completing the table below: Name of the assessment programme: Scope (e.g regional, national or international) National Coverage: Population-based or sample-based Sample Grade level: primary (e.g. grade 1, 2 ) or secondary Secondary Grade 8 (i.e.grade 7, 8) Domains (e.g. reading, mathematics) Language (Reading & Writing), Mathematics & Science Contextual/background questionnaire (e.g. Student, Teahcher & HT (HT questionnaire focus on student, school, teacher or parents) School related data)

Guiding questions • Q.2 What kinds of analysis (e.g. descriptive, correlational or multi-level) do you carry out to identify the factors associated with learning outcome? – Descriptive & Correlational. • Q.3 What are the issues and challenges in doing such analysis? How did you overcome these challenges? – Response from the student found inconsistent and unrealistic – Not able to analyze teacher and HT data due to limited human resources and capacity. – Data cleaning, judgement, data omitting, recoding are the measure to overcome challenges.