notes Developing CITE, a Concept Inventory Tool for Electrical Engineering Eva Fjällström Steffi Knorn Damiano Varagnolo Kjell Staffas 1
A schematic representation of a generic university program notes courses 1 2 3 4 5 6 7 8 9 10 11 12 a b A whole program can be thought as a series of courses, each introducing, extending or building concepts on top of specific concepts c d e 2
A schematic representation of a generic university program notes courses 1 2 3 4 5 6 7 8 9 10 11 12 a b For example the first course may introduce two concepts, in this case a and c concepts c d e 2
A schematic representation of a generic university program notes courses 1 2 3 4 5 6 7 8 9 10 11 12 a b The second course may introduce concept b but ladder or extend a concepts c d e 2
A schematic representation of a generic university program notes courses 1 2 3 4 5 6 7 8 9 10 11 12 a b concepts . . . and so on . . . c d e 2
A schematic representation of a generic university program notes courses 1 2 3 4 5 6 7 8 9 10 11 12 a b concepts . . . and so on . . . c d e 2
A schematic representation of a generic university program notes courses 1 2 3 4 5 6 7 8 9 10 11 12 a b concepts . . . and so on . . . c d e 2
A schematic representation of a generic university program notes courses 1 2 3 4 5 6 7 8 9 10 11 12 a . . . and on. Examinations, though, are typically: • performed at the end of each course b • referring only to that set of concepts that have been introduced or extended in that concepts standard examination strategy: specific course. E.g., here the examination of course 5 will be based on concepts b , c and c at the end of the course & just on e , and thus ignore if students have been forgetting a and d or not; the course material • this is obviously a natural choice. d e 2
A schematic representation of a generic university program notes courses 1 2 3 4 5 6 7 8 9 10 11 12 a b issue 1: are students aware of However this implies a series of issues. . . concepts c their learning progression within their program? d e 2
A schematic representation of a generic university program notes courses 1 2 3 4 5 6 7 8 9 10 11 12 a b issue 2: are teachers aware of However this implies a series of issues. . . concepts c what students know and which concepts need some review? d e 2
A schematic representation of a generic university program notes courses 1 2 3 4 5 6 7 8 9 10 11 12 a issue 3: does the program board have numerical data for: b ● monitoring and evaluating the However this implies a series of issues. . . concepts students’ progression in relation to c the program goals? ● implementing evidence-based d changes on course- and program levels? e 2
A schematic representation of a generic university program notes courses 1 2 3 4 5 6 7 8 9 10 11 12 a how can we solve b Even more importantly, there is no systematic strategy at the program level for collecting evidence these issues? on how much and how fast students forget each individual concept. And without numerical concepts evidence the problem-correction actions risk to be opinion-based instead of being facts-based. c our intuition: collect and process d complementary data e 2
Ideal situation (i.e., best possible one in terms of amount of information) notes Unrealistic assumption: every month we can measure the knowledge level of each student on the whole program (this assumption will be removed later on) 10 of student XXX From mathematical perspectives the best thing possible would be to have a situation like the mastery level one assumed here 5 a bc de 0 1 2 3 4 5 6 7 8 9 10 concept month 3
Ideal situation (i.e., best possible one in terms of amount of information) notes Unrealistic assumption: every month we can measure the knowledge level of each student on the whole program (this assumption will be removed later on) 10 Graphically speaking, we may think that at month 1 we measure this specific knowledge level for of student XXX mastery level a specific student for all the various concepts of the program board 5 a bc de 0 1 2 3 4 5 6 7 8 9 10 concept month 3
Ideal situation (i.e., best possible one in terms of amount of information) notes Unrealistic assumption: every month we can measure the knowledge level of each student on the whole program (this assumption will be removed later on) 10 of student XXX mastery level Then after one month we measure this. . . 5 a bc de 0 1 2 3 4 5 6 7 8 9 10 concept month 3
Ideal situation (i.e., best possible one in terms of amount of information) notes Unrealistic assumption: every month we can measure the knowledge level of each student on the whole program (this assumption will be removed later on) 10 of student XXX mastery level Then after an other month we measure this. . . 5 a bc de 0 1 2 3 4 5 6 7 8 9 10 concept month 3
Ideal situation (i.e., best possible one in terms of amount of information) notes Unrealistic assumption: every month we can measure the knowledge level of each student on the whole program (this assumption will be removed later on) 10 Continuing like this we would get the whole “surface” of how the knowledge is evolving in time of student XXX mastery level for that specific student 5 a bc de 0 1 2 3 4 5 6 7 8 9 10 concept time / course 3
Ideal situation (i.e., best possible one in terms of amount of information) notes Unrealistic assumption: every month we can measure the knowledge level of each student on the whole program (this assumption will be removed later on) But having this information for all the students, one would also have the average behavior, the student A student B student C standard deviations, etc., and have a lot of information that could be used in a lot of different ways. 3
Ideal situation (i.e., best possible one in terms of amount of information) notes Unrealistic assumption: every month we can measure the knowledge level of each student on the whole program (this assumption will be removed later on) But having this information for all the students, one would also have the average behavior, the student A student B student C standard deviations, etc., and have a lot of information that could be used in a lot of different ways. how could we use this data? 3
Information that we could extract - students side notes student A now The test that we want to develop should help students answering these questions . . . do I know enough to take the next courses? how am I currently doing with respect to the program goals? what do I tend to forget, and how fast? where should I focus? what should I eventually know at the end of the program? 4
Information that we could extract - teachers side notes what can I expect my students to know? which variety exists in the prior knowledge among students and across the various years? where shall I start teaching? how shall I adapt to this specific class? . . . and help teachers answering these questions . . . what, how much & how fast do students forget what I teach? (if having comparative data) how do changes in how I teach affect long term performances? (if having comparative data) what are the best changes to do? 5
Information that we could extract - program boards side notes how does the performance vary among students, also across the various years? is there correlation / causation among performances of different courses? is it better to teach a course before or after an other one? where and when students drop? . . . and program boards answering these questions (if having historical / comparative data) how do changes in the program / teaching strategies transform into long term effects in the performances? (if having comparative data) what are the best changes to do? 6
Information that we could extract - program boards side notes how does the performance vary among students, also across the various years? is there correlation / causation among performances of different courses? is it better to teach a course before or after an other one? where and when students drop? . . . and program boards answering these questions (if having historical / comparative data) how do changes in the program / teaching strategies transform into long term effects in the performances? (if having comparative data) what are the best changes to do? 6
notes student A student B student C the main message is this one first message: having this data would be very useful 7
How do we collect this data? notes CITE, a Concept Inventory Tool Of course measuring the knowledge level of every student every month seems infeasible. Nonethe- less testing strategies that are not as detailed as the one above may still provide information useful for Electrical Engineering for everybody 8
How do we collect this data? notes CITE, a Concept Inventory Tool Thus we want to develop a test that can be implemented, and that –even if it is an approximation of the ideal test– gives information useful to everybody. And which kind of purpose would we for Electrical Engineering like to serve? 8
First point: which knowledge shall be assesssed? notes The first point is understanding how these tests should look like 9
Recommend
More recommend
