Moving Forward: Lessons Learned From The Last 10 Years of Risk Modeling
Presentation Agenda 1) Provide a framework for defining risk 2) Examine the role of theory and personal bias in the modeling process 3) Help outline possible partnerships needed for successful processes (data extraction, intervention design, student outreach) 4) Review a list of data sources, variables and tools needed for successful program implementation 5) Guide participants in identifying next steps in their own Risk Identification Programs 6) Examine the impact individualized reports and ad-hoc reports play in day to day lives of advising administrators
Retention Rate: 89.7% Graduation Rates: (2012 ● Top-tier public research university Cohort) ● Founded by Russell H. Conwell in 1884 ● Located in Philadelphia, PA 4-year: 48% ● Average Combined SAT (1170) 6-year: 70% ● Percent Admitted (79.9%) ● Undergraduate Enrollment: 29,416
Framework: Defining Risk ● At risk for what? ● Defining Retention and Academic Success ● 20% Rule
Data Mining Definition 70 - 90% - Spent Cleaning and Understanding data
The Role of Theory and Personal Bias in the Modeling Process ● Dataset Selection ● Variable Inclusion & Exclusion ● Tinto, Astin, Chicoring + Others
It Takes a Village (campus) Business Intelligence Institutional Research Banner Competency Center Colleges & Advising Offices Housing Office Disability Resource Center Registrar Office Office of Orientation Career Center
Tools ● Cognos ● Query Studio ● Tableau ● SPSS (Modeler) ● Good Old Friend Excel ● Canvas LMS
Past Regression Model
Present - Existing Structure 4 Models updated each year for all newly admitted Freshman & Transfer students ○ Admission - Placement ○ Orientation - Add/Drop ○ Add-Drop - W date ○ End of Term Present: Configural Analysis (Classification or Segmentation)
Databases Variables Used - Student Life Cycle ○ Frozen Cohort Files ○ Admissions - (High School Info, demographic) ○ College/Major (Change of either) ○ Financial Aid (EFC, Pell, Unmet Need) ○ New Student Questionnaire (Q1 - Q83) ○ Placement Scores and Levels (Eng, Math, Aleks Sub Scores, SATs) ○ Orientation Information ○ Housing + LLC ○ Term Balance (Start - #of Hrs Registered & End of Term), T1 - T8 ○ Mid Semester Grades/Alerts (Cumulative) ○ Hold Indicators, Multiple W Grades ○ Special Populations ○ Flyin4 Participation ○ Delta GPA Term 1 GPA - HS GPA * Most Predictive
Models and Key Variables Model 1: Admissions - NSQ Placement (June 15th) ● Admit Type ● # of Days between Deposit and First Day of Class or September 1 (Calculated) ● # of Days between Orientation and First Day of Class or September 1 (Calculated) ● Race* ● HS GPA, Rank, Percentile, Size of HS ● HS Code (GPA of all students who went to it before) ● Admissions Ranking ● Unmet Need/Offer Amount ● Academic Major/Program* ● Parental Education ● Admissions Letter Code* ● SAT/ACT ● NSQ ~ 15 Variables ● Placement ( Math, ENG, Aleks Score)
Models and Key Variables Model 2: Orientation - Registration - Change of College/Major (September 12th) ● Census Date - Day after Add/Drop period ends ● Change in college since Orientation (Y/N) as of Census Date ● # of Credits Registered (FT/PT) ● Date of Orientation (Late Orientation for TR) ● LLC vs Non LLC ● Balance as of Add/Drop ● Housing assignment (Name of the Building) ● Unmet Need ● EFC ● Pell Amount
Models and Key Variables Model 3: First Semester Half Mark (6 Weeks) ● 1 week after the “W” date ● # of UG Mid semester Alerts ● # of W grades (1 week after 2 date) ● Balance as of W date ● EFC & Unmet Need ● Total Amount Borrowed ● Student’s College/Major ● Special Populations
Models and Key Variables Model 4: End of Term 1 Marker ● January 2nd ● Hold Indicator ● Term 1 GPA (Academic Standing) ● GPA Drop HS - Term 1 (1.25 drop from HS, Honors = .85) Balance as of Final Grades > $200 ● Housing assignment On or Off ● #of Credits Passed <12 ● Math & Eng Grades ● # of UG MidSemester Alerts
Risk Liaison Intervention Training Summer Meetings – July 11th and August 9th July: The Ones that Got Away: Canvas Review, Risk List Review & Group Introductions August: Designing Fall Interventions and Strategic Communications for students at Risk, on Probation and on Warning Fall Meetings (Fourth Wednesday of the Month @3PM) September Working with Data & Final Freshman & Transfer Risk List Review (Bring your laptop and Excel) October CSRDE Comparative Reports Review & Retention Workshop November Failed to Launch: Mid-Semester Alerts, LOA, WE Students December Designing Spring Interventions and Strategic Communications for students at Risk and on Warning Spring Meetings - Fourth Wednesday @3PM February The Murky Middle 2.0 – 2.5 March Intervention Discussion (What Worked & What Didn’t) April No Meeting May Intervention Assessment
Interventions Addition of mentoring and peer mentoring supports Specialized Workshops (Financial Aid, Housing, etc) Mid Semester Initiatives (Letters, phone outreach) Outreach is done by Financial Aid, Housing, DRS, other supporting units not just advising Jobs on campus and work study positions ○ (Need to identify students eligible earlier)
Interventions Advisor ratios review Specialized advisors focused on riskiest of students Advisor intervention design and data training Shift from intrusive to developmental & appreciative advising Shift from group to individual advising for riskiest of students
Retention Canvas Site
Advising Director Feedback ● Results of Qualitative interviews with 12 Schools/Colleges ● Qs Asked ○ How have you used Risk List? ○ What challenges have you faced in working with at-Risk students? ○ What data/reports would make your life easier? ○ What were the most successful interventions done by your department?
Future - Individualized Risk Models Additional Information to be added to the models ● Retention Defined by College (i.e. Retained in Engineering not TU) ● Special Populations Tracked and Monitored Off/On Campus, first gen, low income, adult 24+, LLC, Pre-Business, Pre-Health, Undeclared a. with 60+ credits) ● Examining impact of Grades/Mid Semester Alerts in specific Courses ○ (First Semester ENG & MATH, STAT, ENG, etc.) ● Focus on Freshman, Sophomore & Transfer ● Focus on Specific Majors (Biology, Undeclared (all colleges), Psychology, Biochemistry, Business Management, Pre-Pharmacy, Communication Studies, Computer Science, Early Childhood Education, Engineering, Political Science) ● June, September, January review of models and at-risk student lists/dashboards ● Monthly Advisor/Risk Liaison Workshops
Examining Internal University Student Migration
Examining Program Level Retention and Migration
CSRDE - National Student Consortium for Reports Webinars and Advisor Training Comparative data
Future Partnerships
Questions Alexandra Yanovski-Bowers Assistant Director for Undergraduate Strategic Initiatives Office of the Vice Provost for Undergraduate Studies Temple University 500 Conwell Hall 1801 N. Broad Street Philadelphia, PA 19122 ayanovski@temple.edu p: 215-204-7596 https://www.linkedin.com/in/alexyanovski
Recommend
More recommend
