10 ‐ 13 September, 2018, Penang, Malaysia Capacity development workshop “Conceptualization, Measurement and Use of Contextual Data” Integrating national items into questionnaires of international large ‐ scale studies: The case of Iran Dr. Masoud Kabiri 1
WHAT ARE TIMSS AND PIRLS ASK IN THEIR BACKGROUND QUESTIONNAIRES? 2
Objectives of designing background questionnaires • Gathering information about major factors affecting on students achievement • Accessing to priori information to better estimate the students achievement: estimation Plausible Values in Bayes theorem 3
The leading Questions • What should students learn? • Who provides the instruction? • How is the instruction organized? • Where does instruction take place? • When does instruction take place? • What have students learned? 4
Model of Potential Educational Experiences (Chen & Hsieh, 2008) National/Regional Official System System curriculum goals Teacher characteristics ( intended Certification 1.Tracking curriculum ) 2.Grade level qualifications Student 3.Content characteristics Teacher decisions 1.background School School goals professional 4.Related 2.Household organization & characteristics economic capital environment 3.Household School course cultural capital Teachers’ offering & Teacher 4.Attitude content goals instructional 5.Activities Classroom characteristics ( implemented support functions 6.expectations a.Background curriculum ) b.Subject matter Instructional TIMSS test orientation activities outcomes c.Pedagogical Implemented Attained Student beliefs curriculum curriculum What are students Who delivers the How is instruction What have expected to learn? instructions? organized? students learned? 5
Conceptual framework of determinants of student outcomes (Nilsen, Gustafsson, and Blömeke) 6
Various aspects of items in BQ in TIMSS • Types of background questionnaires in TIMSS and PIRLS: – Student questionnaire – Home questionnaire (grade 4 only) – Teacher questionnaire (for grade 8 math and science questionnaires, seperatly) – School questionnaire 7
Student Background questionnaire • Contains the responses of the students – Attitude towards the subject measured – Personal/academic backgrounds – School experiences – Career/educational goals – Social ‐ economic statues (SES) 8
Teacher Background questionnaire • Contains responses of the teachers – Background and education – Instructional practices – Academic climate of school – Resources for Teaching – Homework 9
School Background questionnaire • Contains the responses of the school principals – School Enrollment and Characteristics – School climate – Availability of resources 10
Home Background questionnaire • Contains the responses of the parents of tested students to home background – Early literacy activities – Home educational resources – Language in the home – Out ‐ of ‐ school (literacy) activities 11
WHAT DID WE WANT TO ADD INTO TIMSS/PIRLS QUESTIONNAIRES? 12
Problems in TIMSS and PIRLS questionnaires • International questionnaires collect comprehensive, but: – It collected to explain leaning of students, but not necessarily policy making about learning – Amount of asked information is limited relatively to other questionnaires (especially in school questionnaire) – Some of useful information were not covered 13
Reason for importance of adding national items • Gathering information in nationally representative samples • Linking to students achievement • Obtaining baseline to record change (in the case od repeating the items) • No more expense to gather data 14
Challenges of adding items (Limitations) • Should not change the context of questionnaires • Should not be so many that respondents being tired and refuse to response either whole or a part of questionnaire • Should not enter meanwhile of items, they should add at the end of questionnaire • Should be presented like the international items (in content and layout) • Adapting the codebooks and NAFs in order to integrate new questions 15
How selecting the items • Analyzing the TIMSS/PIRLS background questionnaire to find gap informations • Interview with some policymakers • Review background questionnaire of major large ‐ scale assessment: PISA, TALIS, NAEP, TEDS, NALS, ALL, ICCS, ICILS, PIAAC 16
Sample items • School: – Size of school – Enrolment policy – Administrative activities – School authorities • Teacher: – Recruitment mode – Professional development – Assessment and evaluation in school – Attitudes to textbooks • Parent: – Educational costs for children – Cooperation with school – Spent time with children • Student – Working with calculator 17
Recommend
More recommend
