Seeing the Future: Using Predictive Analytics to Gain an Advantage in the Changing Environment
Why is this important? … Proper and proportionate allocation of resources is more important than ever!
Data Data used for the analysis: • Allegheny County HealthChoices and ‘Base’ data from 2005 to 2015; including: • Claims • Demographics • Environmental Variables • Calculated Variables
What is a High Utilizer? A client on w hom you spend a disproportionate am ount of resources There are many ways to define a high utilizer Frequent IPMH Stays • Frequent Re-admissions • High-cost service users • High-unit service users • People frequently in a crisis • For this project, we have chosen to define high utilizers based on their overall annual paid claims amount . This definition allows us to look at individuals from year to year. It also allows us to see if/how they move in and out of being a high utilizer over time.
Defining High Utilizers Pareto principle (80-20 rule) 80% of cost is incurred by 20% of the population • This holds true with the Medicaid BH population •
Question? Although 80-20 m ay apply in a given year, is there m em bership stability in this ratio over tim e (i.e. Are they the sam e people from year to year)? …NO! They are not the sam e!
Movement over time The number of high utilizers from year to year remains consistent; however, Each year, 44% of the high utilizers from the previous year drop out of high utilization for the current year and their place is filled by people who were not high utilizers the previous year! So, if you only looked at last year’s High Utilizers to determine allocation of resources, you would be “wrong” 44% of the time! Our analysis of individuals over time has also shown that there are different “types” of high utilizers. We have grouped them as follows: Career • Chronic • Moderate • Sporadic • Never •
“Types” of high utilizers Career – people who have been high utilizers every year of their tenure in the system Chronic – people who have been high utilizers 75% to 99% of their years in the system Moderate – people who have been high utilizers 25% to 74% of their years in the system Sporadic – people who have been high utilizers 1% to 24% of their years in the system Never – people who have had zero high utilizer years up to this point This realization helped inform our definition and our decision to use the Top 5% cut point. This gives us a mix of types of high utilizers in our “target” group. For example, the Top 1% was populated mostly by the “Career” high utilizers, but for our project this was not ideal.
Can w e predict w ho is going to be a high utilizer next year? … Yes, with a lot of math
What is Predictive Analytics? Predictive analytics uses many techniques from data mining, statistics, modeling, machine learning, and artificial intelligence to analyze current data to make predictions about future. This process can be used to: Find relationships between variables • Exploit patterns found in historical data • Inform decision making • Identify opportunities/risks • Assist in forecasting • Classifying groups based on common characteristics •
Examples you may be familiar with? • Amazon – What other books you may like. • Netflix – If you liked this movie, you will probably like this one also • FICO score – Will you pay your loan back? … And many, many more!
Predictive Analytics Process
Model Accuracy There are many methods for measuring model accuracy. Lift • Gains • ROI • Receiver Operating Characteristic (ROC) Curve • Contingency Table • Methods used vary depending on the purpose of the model. Models are going to be WRONG , the key is to minimize the effects of those errors and to control where you are wrong .
Model Accuracy Contingency Table This method can help you identify where you are correct and where you are incorrect. Models can be tweaked to minimize certain types of errors, and the affects can be seen easily through the contingency table.
What if we had no model? Total People: 12,984 High Util: 1,029 Non-High Util: 11,955 False Positive Rate: 50% True Positive Rate: 50% 515 Correct: 6,493 (50%) 514 Positive Predictive Value: 7.9% Negative Predictive Value: 92.0% Total “Yes” Predictions: 6,493 Total “No” Predictions: 6,493 5,977 5,978
Our Model Total People: 12,984 High Util: 1,029 Non-High Util: 11,955 False Positive Rate: 1.9% True Positive Rate: 40.1% Correct: 12,138 (93.48%) 413 616 Positive Predictive Value: 64.2% Negative Predictive Value: 95.0% Total “Yes” Predictions: 643 Total “No” Predictions: 12,341 230 11,725
Our Model ROC Curve This curve allows us to optimize our prediction cut-points to suit our needs. The farther along the x-axis you travel, the more True Positives you get, but you also get more False Positives. 86.6% AUC AUC – 86.6%
We have a good m odel and have generated predictions for each person…now w hat?! … Decision Theory!!
Informed Decision Making Decision Theory is concerned with identifying the best decision to make, based on a set of assumptions . Our assumption is that the model is correct in its predictions, and that we are going to act on the predictions generated by the model. There are many strategies that can be used in decision theory, but for our case, we are going to perform some sort of intervention for each person that the model predicts to be a high utilizer in the coming year. A combination of predictive modeling, decision theory, and some math can even tell us for how much money we need to be able to perform that intervention.
Informed Decision Making Using our model, we would perform an intervention on 643 people. Of that group, we will be “wasting” effort on 230 of them. For people predicted NOT to be high utilizers in 413 616 the coming year, we will do nothing differently than we did before. They will continue to receive services as they have in the past. 230 11,725
Informed Decision Making With no model, we would perform an intervention on 6,493 people. Of that group, we will be “wasting” effort on 5,977 of them. 515 514 We would also catch 102 more True Positives than with our model. Tweaking the model based on driving factors will help align the decision making process with your goals. 5,977 5,978
What does this look like w hen you add m oney into the equation? Here’s an exam ple!
Informed Decision Making Let’s assume: You see 10,000 people annually • High utilizer ratio is 8% of people incur 60% • of the cost. Those 10,000 people incur $50m in cost • With no model (flipping a coin), you would have 5,000 people that would be predicted to be high utilizers. That leaves you with $5,000 per person per year to perform some sort of intervention
Informed Decision Making Let’s assume: You see 10,000 people annually • High utilizer ratio is 8% of people incur 60% • of the cost. Those 10,000 people incur $50m in cost • With our model, you would have 495 people that would be predicted to be high utilizers. That leaves you with $25,000 per person per year to perform some sort of intervention
Things to be aware of … • This is just a tool • It is not a replacement for clinical expertise • Accuracy is related to timeliness of original data
How can this be used? • Risk Management • Quality Management • Outcomes Management • Case Management • Population Management And Others …
Next steps • Piloting with partners using an automated delivery of longitudinal predictions • Development of additional models to predict IPMH Readmissions, Suicide Attempts, etc. • Discussions with partners to find more useful targets to predict
Recommend
More recommend
