Is Academic Development Ready for Academic Analytics? Using Academic Analytics to Improve Teaching and Learning Brad Wuetherick Executive Director, Learning and Teaching 1 1
Tansi … Tawow Kwe … Pjila’si I would like to acknowledge that we are gathered on the traditional and ancestral territory of the Cherokee and Creek peoples. June 3, 2014 | presented by Jane Smith PRESENTATION TITLE
Dalhousie University Halifax, NS, Canada Medical-Doctoral Research (member of U15) 19,000+ students (~15,000 undergrad) 1000+ faculty 13 faculties -- 180+ degree programs June 3, 2014 | presented by Jane Smith PRESENTATION TITLE
June 3, 2014 | presented by Jane Smith PRESENTATION TITLE
• On a scale of 1-10, where are you: 1 – I know nothing ………………………………………………… 10 – I am an analytics about analytics expert • On a scale of 1-10, where is your Centre: 1 – We are not involved ………………………………………… 10 – We lead analytics in analytics on my campus June 3, 2014 | presented by Jane Smith PRESENTATION TITLE
Academic Analytics A plethora of data about learners + Tools to analyse, cluster, model and predict Deeper personalized information about learners June 3, 2014 | presented by Jane Smith PRESENTATION TITLE
How are Academic Analytics (Learning Analytics) being used? • Improve administrative data for strategic enrolment management • Provide personalized support, inform holistic advising and early alerts initiatives • Improve quality of communication between learners, teachers, and advisors. • Guide and inform course and program design • Improve quality and accuracy of student assessment & program evaluation June 3, 2014 | presented by Jane Smith PRESENTATION TITLE
Case Example #1: John N. Gardner Institute Gateway Courses Initiative https://www.jngi.org/ Looked at the first year American History gateway course across 32 US universities The average D/F/W/I rate for students was 25.5% A. Koch, "Many Thousands Failed" https://www.historians.org/publications-and-directories/perspectives-on-history/may-2017/many- thousands-failed-a-wakeup-call-to-history-educators June 3, 2014 | presented by Jane Smith PRESENTATION TITLE
When institutional demographic data is added, does the conversation change? June 3, 2014 | presented by Jane Smith PRESENTATION TITLE
Faculty Perceptions of Academic Analytics • several studies report faculty skepticism and uncertainty about using such data to inform changes to teaching, learning, and curriculum practices (Andrade, 2011; Dykoff, 2011; Parry, 2012) June 3, 2014 | presented by Jane Smith PRESENTATION TITLE
Faculty Perceptions of Academic Analytics 1. Faculty skepticism about the motivations behind the initiative June 3, 2014 | presented by Jane Smith PRESENTATION TITLE
Why Academic Analytics? • Increased focus on retention and student success (Campbell, DeBlois, and Oblinger, 2007). • what motivates institutions? • Focus on desire for understanding, developing and sustaining a high quality education to help students towards their individual goals and ambitions • Focus on practical realities that retention and student success impacts - rankings, reputation, recruitment, and revenues June 3, 2014 | presented by Jane Smith PRESENTATION TITLE
Faculty Perceptions of Academic Analytics 1. Faculty skepticism about the motivations behind the initiative 2. Concerns about Ethics and Privacy June 3, 2014 | presented by Jane Smith PRESENTATION TITLE
Ethics and the Privacy of Data • Knowledge of student risk factors can result in bias (even if unintentional) from advisors, instructors, etc. • Profiling can be discriminatory and prejudicial • Students have a right to keep personal information private – and a right to be given appropriate notice about the use of their data for institutional purposes • BUT … Institutions have an ethical responsibility to act in the best interest of students based on the data they gather June 3, 2014 | presented by Jane Smith PRESENTATION TITLE
Ethics and Academic Analytics Needs of the Learner ‘Balanced’ Students’ Institution’s Approach to rights to responsibility Academic privacy to act Analytics Needs of the Vendor Institution* Interests* June 3, 2014 | presented by Jane Smith PRESENTATION TITLE
Faculty Perceptions of Academic Analytics 1. Faculty skepticism about the motivations behind the initiative 2. Concerns about Ethics and Privacy 3. Data Literacy June 3, 2014 | presented by Jane Smith PRESENTATION TITLE
What is Data Literacy? Data literacy is the ability to collect, manage, evaluate, and apply data; in a critical manner. • Data collection; Data management; Data analysis; Data visualization; Data policy; Data dissemination; Effective and ethical use of data June 3, 2014 | presented by Jane Smith PRESENTATION TITLE
Faculty Perceptions of Academic Analytics 1. Faculty skepticism about the motivations behind the initiative 2. Concerns about Ethics and Privacy 3. Data Literacy 4. Resistance to Change June 3, 2014 | presented by Jane Smith PRESENTATION TITLE
Academic Analytics at Dalhousie Remembering that the vast majority of Dalhousie’s students are successful, we explored: • Retention patterns: Who is leaving? • Retention Analysis: Who is at the highest risk of leaving? • Retention Analysis: Why do students leave? What are the common characteristics of students who leave? • Predictive Modelling: What have we learned to help us support potentially ‘at-risk’ incoming students? June 3, 2014 | presented by Jane Smith PRESENTATION TITLE
Academic Analytics • What Factors Matter: Model A - Academic Preparation 1. Incoming grades (particularly Math and English) 2. Writing Fluency/Organization 3. International baccalaureate (IB) and/or advanced placement (AP) vs traditional high school stream June 3, 2014 | presented by Jane Smith PRESENTATION TITLE
Academic Analytics • What Factors Matter: Model B – Pre-Entry Non- Academic Factors 1. Province/Country of origin 2. Rural/Urban 3. Gender 4. Age 5. Socio-Economic Background 6. Family Educational Background 7. Ethnicity June 3, 2014 | presented by Jane Smith PRESENTATION TITLE
Academic Analytics • What Factors Matter: Model C – Post-Entry Academic and Non-Academic 1. Faculty enrolled 2. High risk (DFW) courses 3. Low SRI courses 4. Not ‘First Choice’ program 5. Residence 6. Loans/Bursaries 7. Awards/Scholarships 8. Varsity Athletics June 3, 2014 | presented by Jane Smith PRESENTATION TITLE
Academic Analytics • What Factors Matter: Model D - Academic Performance/Behaviours 1. Attendance 2. Early Midterm Grades (need to make this systematic) 3. Fall-term GPA 4. Credit Hours Completed 5. Switch to part-time in Winter term June 3, 2014 | presented by Jane Smith PRESENTATION TITLE
Academic Analytics • What Factors Matter: Model E – Surveys (NSSE, CUSC, CSI) 1. High scores on Levels of Academic Challenge (LAC), Supportive Campus Environment (SCE), Active and Collaborative Learning (ACL), and Student-Faculty Interactions (SFI) 2. Campus engagement – positive relationships with other students; helping other students with academic work; participation in co-curricular; living on or close to campus 3. Academic engagement – tutoring other students; receiving prompt feedback; belief they were gaining work-related knowledge and skills June 3, 2014 | presented by Jane Smith PRESENTATION TITLE
Academic Analytics • What Factors Matter: Model F – Learning Activity Data (Analytics from LMS and other Educational Technologies) 1. System-Level Data – course grades, comparative, etc. 2. Individual-Level Data – individual assessment performance, response to individual items, etc. 3. Transaction-level Data – number of times logging on, time on task, ‘click rate’, use of hints/help systems, etc. June 3, 2014 | presented by Jane Smith PRESENTATION TITLE
Case Example #2 Course design consultation in STEM Gateway course – • new to faculty member, historically high D/F/W rate (~30%) Been taught as a traditional 3 hr lecture/3 hr lab • Students without 85% in HS math are more likely to • struggle (3x more likely to receive D/F/W) All students previously rated SFI and ACL (on NSSE) lower • than peers (a known risk factor for retention) Students on Pell Grants were more likely to report that they • did not believe they are gaining work-related knowledge and skills (a known risk factor for retention) Students in past five years who lived in residence were less • likely to receive D/F/W (a protective factor for retention) June 3, 2014 | presented by Jane Smith PRESENTATION TITLE
Case Example #2 2 months before the term starts, you meet with the faculty • member Institutional research office was able to send the following • update: 62% of this year’s cohort has below 85% in HS math 33% of the students are in residence 68% of the students are on Pell Grants June 3, 2014 | presented by Jane Smith PRESENTATION TITLE
Recommend
More recommend
