Identifying and understanding 21st Century Skills Professor Jeppe - PowerPoint PPT Presentation

Professor Jeppe Bundsgaard Danish School of Education Aarhus University Identifying and understanding 21st Century Skills Professor Jeppe Bundsgaard

Outline ● 21 st Century Skills projects – What is it? And what are the problems? ● An alternative approach to identifying 21 st skills – Prototypical situation oriented approach ● How to identify and understand 21 st century skills as part of assessment development – The Danish 21 st Skills assessment Professor Jeppe Bundsgaard

A number of 21 st Century Skills projects P21 Partnership for 21st Century Skills ATC21s WEF Professor Jeppe Bundsgaard Assessment and teaching of 21st Future of Jobs Century Skills

Methods to identify and describe 21 st Century Skills ● Brainstorming – P21 – “P21 has identified and brought to the forefront a comprehensive set of skills that, along with content mastery, are what all sectors can agree are essential for success. The P21 Framework for 21st Century Learning took several years to develop ...” ● Survey of frameworks – ATC21s – “To arrive at this model framework we compared a number of available curriculum and assessment frameworks for twenty-first century skills and skills that have been developed around the world. We analyzed these frameworks to determine not only the extent to which they differ but also the extent to which these frameworks provide descriptions of twenty-first century learning outcomes in measureable form.” ● Survey of experts – World Economic Forum: The future of jobs – “survey of CHROs and other senior talent and strategy executives of leading global employers” Professor Jeppe Bundsgaard

Challenges ● Identification challenge – How do we know that we have found the core skills? ● Ranking challenge – How do we know that these skills are the most important ones? ● Description challenge – How do we define, describe and subdivide each skill/competence? ● Teaching challenge – How do we know which parts of the skills to focus on? What is hard, what is easy? ● Organizational challenge – How can the disciplines contribute (methods, knowledge, epistemic frames)? How should the school be organized? Professor Jeppe Bundsgaard

Solution? A prototypical situation oriented approach ● A prototypical situation is a central member of a category of situations (Cf. Eleanor Rosch) ● Recurrent situations, typified rhetorical action (Miller) ● E.g. – Being a teacher means participating in ● Teaching situations ● Parent-teacher conferences ● Professional development ● Eating lunch ● Walking the hallways to class – Not all these situations are prototypical of being a teacher. Professor Jeppe Bundsgaard

The method ● Politics ● Identify – (first very general, later on ● Sociology more specific) prototypical situations which we want ● Anthropology students to be able to take part in ● ‘Didactology’ ● Analyze challenges ● Psychology – Describe and examine empirical practices ● Identify competences (skills) – Analyze the challenges and the ways to address these challenges, i.e. the competences needed Professor Jeppe Bundsgaard

How to identify prototypical situations ● Through sociological reasoning – What are the main domains of a society? ● E.g. Habermas: The civil society – What are the main types of situations in these domains? ● Through sociological imagination – How will our societies develop? – What will be the main types of situations in this imagined future? ● Through political dialogue – What society do we want students to participate in developing? – Which situations should they therefore be able to take part in? Professor Jeppe Bundsgaard

Six core identities and their general prototypical situations Professor Jeppe Bundsgaard

How to identify prototypical situations II, and identify challenges ● Through empirical studies of main types of situations – E.g. Studies of prototypical work practices (cf. Wenger) – Surveys, interviews, observations ● Through sociological and anthropological reasoning and imagination ● Through production and study of fiction (literature, movies, poetry etc.) Professor Jeppe Bundsgaard

An example Danish L1 Teacher’s prototypical situations Professor Jeppe Bundsgaard

How to identify competences? ● Analyses of how challenges are met/could be met ● Didactical and psychological reasoning and imagination. ● Identification of approaches, methods, knowledge etc. from the academic disciplines – And how they can be brought into use. Professor Jeppe Bundsgaard

What are the most important skills? (the ranking challenge) ● Different criteria: – The situations that most people are in (or is expected to be in), most of the time (sociology/anthropology) – The situations we as a society find most important to improve on (policy) ● As a citizen ● As a worker ● As a person, consumer, aesthete, etc. ● Which criteria are used in the 21 st skills projects? Professor Jeppe Bundsgaard

The description and teaching challenges ● An example: – An assessment of 21 st Century Skills Professor Jeppe Bundsgaard

Design principles ● Design six scenarios and story lines ● Put the students in prototypical 21 st century situations in these scenarios ● Give the students tasks that are prototypical of the situations – With a focus on handling and production of information, collaboration, project work Professor Jeppe Bundsgaard

2 example modules Professor Jeppe Bundsgaard

We are going to get a pet in the after-school club Professor Jeppe Bundsgaard

Brainstorming ideas Professor Jeppe Bundsgaard

Reflecting on other peoples suggestions Professor Jeppe Bundsgaard

Researching the web Professor Jeppe Bundsgaard

Planning how to build a cage Professor Jeppe Bundsgaard

Choosing the design Professor Jeppe Bundsgaard

Finding information Professor Jeppe Bundsgaard

Evaluating information Professor Jeppe Bundsgaard

Planning how to take care of the rabbit Professor Jeppe Bundsgaard

Designing a poster Professor Jeppe Bundsgaard

Charity dinner Professor Jeppe Bundsgaard

Brainstorm tasks Professor Jeppe Bundsgaard

Allocate tasks Professor Jeppe Bundsgaard

Plan the preparations for the event Professor Jeppe Bundsgaard

Interior design Professor Jeppe Bundsgaard

Search information Professor Jeppe Bundsgaard

Evaluate information Professor Jeppe Bundsgaard

Evaluate information Professor Jeppe Bundsgaard

Collaborate and solve conflicts Professor Jeppe Bundsgaard

Collaborate with external business partners Professor Jeppe Bundsgaard

Prepare a budget Professor Jeppe Bundsgaard

Design a web page Professor Jeppe Bundsgaard

Design ● Around 3000 students from 3 rd through 8 th grade in 15 schools took the test – Pre-test winter 2013 – Post-test spring 2015 ● 6 modules ● Each student was assigned two modules at pre- test and two other modules at post-test Professor Jeppe Bundsgaard

Rotated design Module A B C D E F Grades Pre Post 3rd Post Pre Pre Post Pre Post Pre Post Post Pre 4 th – 6 th Post Pre Pre Post Post Pre Post Pre Pre Post Pre Post Pre Post Post Pre 7 th – 8 th Post Pre Pre Post Post Pre Post Pre Professor Jeppe Bundsgaard

Dimensions ● Scenario competence – Imagine complex situations and envisage development and solutions to challenges ● Process competence – Planning and addressing issues that occur during a project ● Composition competence – Compose multimodal messages that fits the situation ● Information competence – Prepare and carry through Internet searches, and be critical towards the information ● Collaboration competence – Communicate and collaborate with peers, experts, costumers etc., identify and solve conflicts Professor Jeppe Bundsgaard

Short digression Rasch analysis helped my understanding ● Scenario competence ● Each task was described by a dimension and a subdimension – Brainstorm ● Scenario competence kept having bad fit – Creating ideas ● It turned out that most bad items – Planning was from the planning subdimension – Consequences of ● I created a new dimension choices mostly consisting of these items: – Interior design the process dimension. ● An indication that management – Systematize of time is not strongly connected to the management of actors and space Professor Jeppe Bundsgaard

Scoring ● Multiple choice ● Drag and drop ● Human scored short text responses ● Human scored multimodal responses ● Polytomous and dichotomous items Professor Jeppe Bundsgaard

Data ● 2604 cases (pre-test – post-test for 1302 students) ● 231 variables – Collaboration: 18 – Information: 29 – Production: 51 – Scenario: 60 – Process: 73 Professor Jeppe Bundsgaard

Rasch analysis ● Polytomous facet model ● Using grade as a categorical variable to account for the non-normal distribution of students ● Using gender to account for DIF Professor Jeppe Bundsgaard

Reliability, FIT ● Fit: Weighted MNSQ between .75-1.21 ● PV-reliability: 0.679, 0.621, 0.681, 0.816, 0.677 Professor Jeppe Bundsgaard

Gender and Grade differences Professor Jeppe Bundsgaard