theory to application Judy Cohen** & Catherine Robinson* **UELT - PowerPoint PPT Presentation

The UK’s European university Team-Based Learning in Practice: Moving from theory to application Judy Cohen** & Catherine Robinson* **UELT *Kent Business School Chatham, Kent Thursday 4 May 2017, 10am -1.30pm, Medway Campus, Rochester R2.09

Introduction • Firstly, an apology …. • I’m here to provide a practical take on implementation in Kent • What led us to TBL as an instrument of change? • What were the challenges faced? • What were/are the results (so far)? • Next steps?

The Scenario: (Macro) Economics for Business • Stage 2 students • Compulsory for A&M, optional for B&M • Increased cohort • 35 in 2014 • 76 in 2015 • 109 in 2016 • ‘Under - performance’ • Low lecture attendance levels (as low as 20%) • Poor engagement in seminars (attendance monitored) • Relatively weak assessment scores (mean ~57%) • Satisfaction scores acceptable (4.2/5) CB363 – TBL in context Page 3

Module Learning objectives • Demonstrate knowledge of the underlying economic concepts, competing theories and basic principles of macroeconomics. • Evaluate the effect of the changing/prevailing macroeconomic environment on business decisions, behaviour and performance. • Interpret macroeconomic data and develop well-structured lines of economic argument, offering critical comment on the arguments of others. • Demonstrate knowledge of the importance of economics in understanding current business issues in the UK, Europe and global economies. • Retrieve information from a variety of sources. • Undertake independent and self-managed learning. • Draw on social science concepts and theories in decision-making situations. • Demonstrate ability to communicate information, ideas and solutions effectively. Page 4 Module LOs

How did we do it? • 110 students split into teams of 6/7 • Named after famous economists • Lecture slots used to test the students’ readiness and the individual scores used as assessments • Continual assessment at 5 points during the 12 week course • Highest 3 scores were taken as their MCQ mark (worth 10% in total) • This was administered using Turningpoint responseware devices, registered to individual students. • 5 MCQs at the start of the hour, followed by breakout time into teams…. Page 5 Implementation

Readiness Assurance Process (2) • Students were then asked to sit in their teams and discuss the same 5 questions, reaching agreement on their answer – Required team debates, decision making, rationalising the chosen answer within the team – Answer revealed via scratchcards – Resulted in immediate feedback from peers and generated continued discussion on more contentious answers – Team results stored and used to award an end of module prize • Opportunity to challenge – Students were able to nominate an individual to challenge the answer to any of the iRATs but Page 6 RAPs

Final component of the lecture hour…. • A recap and correction to any misinterpretation • informed in part by the questions • Extension into the application of the key concepts • Summary of the resources available Page 7

The Application Exercise • Each team was presented with a pack containing: • A Question Sheet • A menu Stand • A pack of 5 MCQ letters (A to E) ● Teams were given 20 minutes to answer the question • Simultaneous reveal essential • Each team tasked with justifying their answer • NO CHANGING YOUR MIND! Page 8

Examples …. • Applications need to meet the 4S criteria (Sibley and Ostafichuk (2014) • Significant; Same; Specific; Simultaneous • Some worked well: • The more structured, shorter questions • The ones with RIGHT answers • Some not so well: Which of the following is likely to have the biggest impact on structural unemployment? Prepare to explain your answer. a) Growth in the third sector (the Big Society) b) Growth in grammar schools c) Growing global inequality d) Growth in Artificial Intelligence e) An aging population Page 9

Challenges faced: INSTITUTIONAL • square pegs, round holes: • Module constraints • Assessment patterns fixed and not easily amended • Timetable constraints • Traditional Lecture – seminar format • Students can change seminar slots up to 3 weeks into term • Rooming constraints • all KBS lectures are in RHDC and seminar rooms are laid out in rows • Technological constraints – Moodle-turningpoint interface • Led to disjointed delivery of the complete TBL cycle • Teams were not as cohesive • Administration was time consuming Page 10 Challenges

Challenges faced: TBL IMPLEMENTATION • Student expectations (herding cats): • Student shuffling teams • Honesty • The RAP was inevitably quite complicated • Split across two single hours meant the TBL cycle was not complete in one sitting • Small seminars actually worked against creating a competitive environment • The compilation of the data was tremendously painstaking • Peer review • How to incorporate it? • What was it for? Page 11

The Royal Historic Dockyard Church Page 12 Teaching Environment

The Royal Historic Dockyard Church Page 13 Teaching Environment 2

So, how did it progress? • Generally students seemed receptive to the idea of TBL • Early feedback indicated they were not liking TBL: • ‘I am not paying £9,000 to teach myself’ • Turningpoint devices were meant to have a plugin to the moodle site. • Data had to be extracted and managed using Excel • Students were using multiple turningpoint devices to register attendance – corrected for by taking a register by hand but all of this involved extra work

Implementation of TBL Requires…. • Selling it to the students • ORGANISATION! • Online materials – Topic books containing slides, articles, videos – Readiness Checkers • Scratchcards – For the t-RATs • Seminar materials – Menu stands, laminated letters • Feedback – (at a variety of places through a number of mechanisms) • Applications – Applied questions that required greater thought on how to use the material – Not always a right answer – the rationale and reasoning was important • Peer evaluation Page 15

Results – Attendance • Did it work? • Elements of it, certainly, despite teething troubles N=109 (2016) and 76(2015) 88% home students in 2016 compared with 83% in 2015 Around 38% female students in both cohorts Lecture attendance this year averaged 72% Accounting and management students (comparative figures for 2015 not available, but made up a higher proportion of students it was not uncommon to have less than 10) in 2015 (around 70% compared to around 50% in 2016) – not shown here Page 16 Results

Performance Final exam results not yet available Regression analysis controls for gender, POS, fee and part time status, as well as seminar attendance Page 17 Performance

Preliminary multivariate results VARIABLES mark cw_mcq cw_ict fees (Home student=1) 0.472 1.538 1.75 (0.568) (2.66) (2.282) Gender (female=1) 0.522 -0.295 2.554 (0.425) (1.975) (1.704) year (2016=1) 1.378*** 14.36*** -0.644 (0.427) (1.972) (1.692) sem_attend (%) 6.131*** 19.06*** 20.52*** (1.028) (4.749) (4.1) deg_type (A&M) 0.173 2.397 -0.569 (0.422) (1.953) (1.682) Constant 13.37*** 48.55*** 43.48*** (1.047) (4.803) (4.148) Observations 175 179 177 R-squared 0.219 0.286 0.144 Standard errors in parentheses *** p<0.01, ** p<0.05, * p<0.1 Page 18 OLS regressions

Any evidence of ‘success’? • Well, the MCQ marks are higher, but this was an incentive to get them to play. • Attendance was necessarily up, but need to see if this is evidenced through improved performance! • The ICT marks were the same as last year, but a subtly different question so hard to evaluate this as an achievement of the TBL method. • What we can say is that module achievement was consistent

Student evaluation – what did they think? • Interim survey • Module evaluation (end) • Focus groups • Teaching quality survey (end of year) Page 20 Student evaluation

Overall student evaluation ..This year Last year.. Page 21 MEQ