Enhancing student project selection and allocation in higher education programmes Johann A. Briffa 1 Simon Lygo-Baker 2 26 September 2018 1 Dept. of Comm. & Computer Engineering, University of Malta, Msida MSD 2080, Malta 2 Dept. of Higher Education, University of Surrey, Guildford GU2 7XH, England 1
Outline
Overview Introduction and motivation Proposed method Results Conclusions 2
Introduction and motivation
Introduction Final year projects – motivation • Most undergraduate degrees include a substantial project in the final year • Allows student to: • Demonstrate command of subject • Integrate material from taught modules • Demonstrate higher order skills (e.g. application, synthesis, evaluation) • Transition towards self-sufficiency • Often the first major piece of work for students • Usually assessed through a report / dissertation 3
Introduction Final year projects – mechanics • Shift in emphasis towards a research-led approach • Involves working on a specific topic for an extended duration (beyond standard coursework deadlines) • Under supervision of academic staff (or possibly graduate students or post-doctoral fellows) • Work is performed independently but supported by tutor • Relationship between learner and tutor is potentially of great significance • (Much of this also applies to post-graduate degrees) 4
Introduction Allocating supervisor and topic • Need to allocate all students to a finite number of supervisor, taking into account: • Topics of interest to student • Topics of expertise of supervisors (esp. at higher levels) • Equitable distribution of students among academics • This can be problematic, as students may have strong preference for choice of supervisor, may be due to: • perceived shared interest in area • topics of interest considered ‘hot’ or ‘easy’ • perceived leniency or helpfulness of academic 5
Introduction Allocating supervisor and topic – common approaches • Coordinator-led method: 1. A list of titles is published, each connected with an academic 2. Students select a list of preferences 3. Coordinator manually allocates a project per student (maybe with a defined order of allocation) • Student-led method: • Students meet potential supervisors and agree on a title • Requires a lot of pre-allocation meeting time • Can overwhelm popular supervisors • First-come-first-served increases pressure on students • Can favour some students (e.g. who live closer) 6
Introduction Other complications • Requests for change of topic / supervisor • Often because supervisor is not ‘preferred’ • Or when the allocated topic does not match expectation • Difficult to solve without repercussions • Reallocation increases imbalance in teaching load • Multiplier effect if students see this as an option • Ideal is to avoid the need as much as possible • In some cases it’s unavoidable (e.g. staff illness) • Having a published approach helps 7
Introduction Other complications • Multiple students working on the same title • Requires clear separation of work • Consideration of dependencies • Collaborative work with third party (e.g. industry) • Often tied to specific academic (pre-established link) • Can also be tied to student (e.g. previous placement) • Formal agreement with University often required • Contingency plan to ensure student completion if industry partner pulls out 8
Introduction Examining projects • Project generally has a significant weighting, so typically is assessed by two examiners • Final mark obtained by some process of agreement (may be simply a weighted average) • Significant discrepancies need to be reconciled • Supervisor may or may not be one of the examiners (depends on institutional policies) 9
Introduction Allocating examiners – constraints • Examiners need to have necessary expertise • Marking may need to be blind in the first instance • May be preferable to avoid consistent examiner pairings (limits potential for problems with agreement) • May need to avoid certain pairings of examiners (e.g. to avoid conflicts of interest, incompatible personalities, or having two harsh or two lenient markers) • Allocation of duties needs to be equitable among staff • If supervisor is not an examiner, need to consider three-way constraints 10
Introduction Administrative overhead • Often handled by a projects coordinator; alternatively by the board of studies or its delegate • Collection of proposed titles from staff • Collection of preferences from students • Allocation of supervisor and examiner(s) • Moderate grade agreement where necessary • Consider complaints and other requests from students and staff 11
Proposed method
Proposed method Key features • Separation of title selection from allocation of supervisor • Title selection in discussion with supervisor after allocation • Allocation of supervisor takes into account topic preferences • Students submit a prioritized list of preferred keywords • Each keyword is associated with one or more academics • Avoids selecting supervisors directly (popularity contest) • Adds flexibility to final allocations 12
Proposed method Key features • Uses an interactive web application to collect preferences • Gives immediate feedback on current keyword popularity • Students with popular choices can make informed choices: 1. Either accept higher risk of not getting top preference 2. Or lower their risk by picking a less popular topic • Allows students to partially self-select, simplifying allocation • Makes process as transparent as possible • Uses a global optimization algorithm to allocate supervisors • Quicker and less laborious than manual process • Easier to ensure that all constraints are satisfied • Treats all students and staff equally, removing potential bias • Likely to come closer to an optimal allocation 13
Proposed method Global optimization algorithm • We express the problem as a simulated annealing problem • Energy function (which we seek to minimise): 1. For each supervisor, increases exponentially with the discrepancy between allocated and nominal load 2. For each student, increases logarithmically as the matched keyword goes down in priority 3. For each student, applies a high fixed penalty if the allocation does not match any keyword • Start with a random allocation and a high ‘temperature’ • Whole process is in the order of a few minutes 14
Proposed method 100 Acceptance 80 Improvement Percentage 60 Example timeline 40 Initial ‘temp.’ high 20 0 enough to accept 10 11 all state changes; 10 10 10 9 10 8 often improves Energy 10 7 10 6 allocation 10 5 10 4 10 3 10 2 10 7 10 6 10 5 10 4 10 3 10 2 10 1 10 0 10 -1 10 -2 Temperature 15
Proposed method 100 Acceptance 80 Improvement Percentage 60 40 Example timeline 20 At final ‘temp.’ few 0 changes accepted; 10 11 10 10 10 9 rarely improves 10 8 Energy 10 7 allocation 10 6 10 5 10 4 10 3 10 2 10 7 10 6 10 5 10 4 10 3 10 2 10 1 10 0 10 -1 10 -2 Temperature 15
Proposed method 100 Acceptance 80 Improvement Percentage 60 40 20 Example timeline 0 Allocation 10 11 10 10 improves in stages 10 9 10 8 Energy 10 7 10 6 10 5 10 4 10 3 10 2 10 7 10 6 10 5 10 4 10 3 10 2 10 1 10 0 10 -1 10 -2 Temperature 15
Results
Results Overview • We give results from two consecutive years • 2012/2013: • Students choose from a list of published titles • Each title associated with an academic • 2013/2014: • Students choose from a list of published keywords • Each keyword associated with at least one academic • Allocation using simulated annealing in both cases 16
Allocation results 2012/2013 Summary for 2012/2013 • 110 students • 19 academics; three reduced load, one increased load • 100 project titles; some could be taken by more than one student, some titles were restricted by programme of study • Students could choose a ‘Title to be agreed with supervisor’ • All but one student submitted their preferences (prioritized list of five titles from at least three academics) • Six students had discussed a collaborative project; these were given priority • Four students indicated a particular concern about supervisor; these were also given priority 17
Allocation results 2012/2013 Preference Number of students Rank 1 33 students Rank 2 12 students Rank 3 8 students Rank 4 10 students Rank 5 9 students Random (preferred) 2 students Random (not preferred) 36 students Notes • Rank 1 includes 10 students allocated as a priority • Random includes 1 student who did not submit preferences 18
Allocation results 2012/2013 Preference Number of students Rank 1 33 students Rank 2 12 students Rank 3 8 students Rank 4 10 students Rank 5 9 students Random (preferred) 2 students Random (not preferred) 36 students Notes • Only 33/110 students allocated their first choice • Supervisor not associated with any preference in 36/110 • Led to considerable student dissatisfaction 18
Recommend
More recommend
