BIG Goals Predictive Analytics: Practical insights into Goals, Means, & Managing the development of an Of your faculty analytics platform Expectations About your students Tony Scinta Nevada State College Key questions to ask: About your institution About your philosophy Nevada middle tier of higher education - circa 1999 Institutional Challenges Inadequate Academic Preparation Nevada middle tier of 100% higher education Commuter - circa 2016 First- generation/No n-cognitive challenges Work obligations/Poo r finances Watch entire season of Breaking Bad in one weekend
Institutional Policy . . . Enforcement of prerequisites . . . Best practices in Gateways Course scheduling . . . teaching & learning . . . to Completion Assessment techniques. . . Academic support services . . . Fail Often . . . Fail Early . . . 1 st semester GPA below 2.0 Fail Often . . . Fail Often . . . Fail Early . . . Fail Early . . . 4-Year Rate 5-Year Rate Graduate Graduate 100 Percent 100 Percent Don't Graduate Don't Graduate
Fail Often . . . Silver Lining Fail Early . . . 6-Year Rate Year-to-Year Retention 19% Higher 16% Higher Graduate 100 Percent Don't Graduate Used Advising Used Tutoring HS GPA < 3.0 Year-to-Year Retention 25% 19% Higher Higher Used Advising Institutional GOAL: Challenges Help students before it is too late MEANS: Early identification of at-risk students Assistance from academic support services
BIG QUESTIONS First Question: What is your goal? What should I ask Second Question: What predicts that goal? before proceeding with an analytics Third Question: How can we know if students effort? are on track to reach the goal? 72 Predictive Model Class Add Date Predicts the probability that a student will earn a grade Transfer Credits HS GPA of C or better in the course GPA 1 st Passed Credits Taken Gender Generation Expected Family Ethnicity 1 st T erm Contribution Academic Major Pass Academic Level Year Attended Ratio Orientation Cumulative Pell Eligible Enrollment GPA Instruction status Mode Remedial Semester Math Distance
GREAT, RIGHT? Usability Pitfalls 1. Student feedback was inadequate o SOLUTION: Added new dashboards Usability Pitfalls 1. Student feedback was inadequate o SOLUTION: Added new dashboards 2. Faculty wanted more data and they wanted it to be more accessible SOLUTIONS: o More data on student cards (e.g., time since last log in) o Emails to faculty o Ability to view grades as raw points or percentages
Modeling Pitfalls Modeling Pitfalls 1. Bad predictions 1. BAD PREDICTIONS! Modeling Pitfalls 1. Bad predictions o Anomalous predictions o Too lenient Low Risk Low Risk Low Risk Low Risk Low Risk Low Risk Low Risk Low Risk 76-100% 76-100% 51-75% 51-75% 51-75% 0-50% 0-50% 0-50% Type II Error? Modeling Pitfalls 1. Bad predictions Number of Students 3500 o Anomalous predictions 3207 o Too lenient 3000 2. Faculty vs. Gradebook 2500 2000 1500 1000 752 356 500 0 Green Yellow Red
Modeling Pitfalls Modeling Pitfalls SOLUTIONS SOLUTIONS Low Risk Low Risk Low Risk Low Risk Low Risk Low Risk Low Risk Low Risk Low Risk 90-100 90-100 90-100 75-89.9% 75-89.9% 75-89.9% 0-74.9% 0-74.9% 0-74.9% Characteristics/History Grades Philosophical Pitfalls “Non-cognitive” Concerns No solution yet Structural Pitfalls
Structural Pitfalls Structural Pitfalls Small Class Size InsufficientAdvisors New Comprehensive Dashboard Structural Pitfalls SOLUTIONS
Take Home Lessons 1. Analytics platforms are not easy to do right o Clarify roles BEFOREHAND o Modeling is not magic • Manage expectations BEFOREHAND • Choose the right parameters 2. One size does not fit all o Our experience – faculty need it less in small courses o Advising may be critical 3. Even done right, there are concerns o Belonging/efficacy should be accounted for o Advising may be critical 4. If it works, it is worth the effort
Recommend
More recommend
