An Automated Forecasting Framework based on Method Recommendation for Seasonal Time Series André Bauer, Marwin Züfle, Johannes Grohmann, Norbert Schmitt, Nikolas Herbst, and Samuel Kounev Chair of Software Engineering, University of Würzburg 11th ACM/SPEC International Conference on Performance Engineering https://se.informatik.uni-wuerzburg.de/
Motivation Fast living and changing requirements • In the course of digitalization, new services and applications will arise • The amount of connected devices will increase Cloud Computing allows flexibility • Complexity exceeds human capacity • “Naive” resource allocation – Consumes 2% of the world‘s electricity – Emits as much CO 2 as the airline industry 1 1 https://e360.yale.edu/features/energy-hogs-can-huge-data-centers-be-made-more-efficient An Automated Forecasting Framework based on Method Recommendation for Seasonal Time Series 2 André Bauer Motivation Related Work Approach Evaluation Conclusion
Problem Visualization Resources shall be supplied • Automatically and Load • In advance Adapt the amount of resources Time • When? • How much? Magnitude Reliable Time Series Forecasting Method An Automated Forecasting Framework based on Method Recommendation for Seasonal Time Series 3 André Bauer Motivation Related Work Approach Evaluation Conclusion
No-Free-Lunch “ No-Free-Lunch Theorem ” [ Wolpert97 ] no globally best forecasting method An Automated Forecasting Framework based on Method Recommendation for Seasonal Time Series 4 André Bauer Motivation Related Work Approach Evaluation Conclusion
Related Work – Hybrid Forecasting Ensemble Forecaster Decomposition Forecasting Recommendation Forecasting - Historically first - Learning a rule set - Combining hybrid forecasting from a set of time advantages of task series different methods - Weighting results - System guesses - Decomposition of from several best forecasting time series into its methods method components - Linear combination - Expert system or - Executing several of these weighted machine learning methods one after results techniques another [ Bates69, Clemen89, [ Collopy92, Wang09, [ Zhang03, Züfle17, Boulegane19, more ] Montero20, more ] Bauer20, more ] An Automated Forecasting Framework based on Method Recommendation for Seasonal Time Series 5 André Bauer Motivation Related Work Approach Evaluation Conclusion
Contributions Research Question I Research Question II Research Question III How to build a hybrid What are suitable time What are suitable forecasting series characteristics approaches for the framework for for the recommendation seasonal time series? recommendation? system? Contribution I Contribution II & III Contribution IV Automated framework New characteristics Evaluation of the for seasonal time and three approaches and series based on recommendation discussion. decomposition. approaches. An Automated Forecasting Framework based on Method Recommendation for Seasonal Time Series 6 André Bauer Motivation Related Work Approach Evaluation Conclusion
Forecasting Framework – Underlying Idea Simplifying time series with logarithm transformation Extracting time series features • Fourier Terms • Season • Trend • Remainder Consulting recommendation system for best suitable machine learning method Forecasting time series • Forecasting each feature with individual methods • Train and predict machine learning method with features An Automated Forecasting Framework based on Method Recommendation for Seasonal Time Series 7 André Bauer Motivation Related Work Approach Evaluation Conclusion
Recommendation Offline Training Time series storage 1. Offline Data Preparation Time series Machine learning characteristics methods evaluation extraction n base-level methods Matrix of Meta-level (n vectors of meta-level data set prediction accuracy) attributes T An Automated Forecasting Framework based on Method Recommendation for Seasonal Time Series 8 André Bauer Motivation Related Work Approach Evaluation Conclusion
Recommendation System A C Time series storage 1. Offline Data Preparation Time series Machine learning characteristics methods evaluation extraction n base-level methods Matrix of Meta-level (n vectors of meta-level data set prediction accuracy) attributes T 3. Online Recommendation Classification Prediction T Best method for each time Random forest model series V 2. Offline Learning An Automated Forecasting Framework based on Method Recommendation for Seasonal Time Series 9 André Bauer Motivation Related Work Approach Evaluation Conclusion
Recommendation System A R Time series storage 1. Offline Data Preparation Time series Machine learning characteristics methods evaluation extraction n base-level methods Matrix of Meta-level (n vectors of meta-level data set prediction accuracy) attributes T 3. Online Recommendation Regression Prediction Rank M n T M 2 M 1 2 … Class 1 n n random forest models Label 1 V (1 for each method) 2. Offline Learning An Automated Forecasting Framework based on Method Recommendation for Seasonal Time Series 10 André Bauer Motivation Related Work Approach Evaluation Conclusion
Recommendation System A H Time series storage 1. Offline Data Preparation Time series Machine learning characteristics methods evaluation extraction n base-level methods Matrix of Meta-level (n vectors of meta-level data set prediction accuracy) attributes T 3. Online Recommendation Prediction Regression Classification M n T M 2 Best method M 1 for each time n random forest models Random forest model series V (1 for each method) 2. Offline Learning An Automated Forecasting Framework based on Method Recommendation for Seasonal Time Series 11 André Bauer Motivation Related Work Approach Evaluation Conclusion
Evaluation Design Machine learning methods: Compared recommendation approaches: • • Catboost [Prokhorenkova18] S* best method a-posteriori • • Cubist [Quinlan93] S L lowest average degradation • • Evtree [Grubinger14] S B most often the best method • • Feed-forward NN [Hyndman14] A C classification approach • • Random forest [Breiman11] A R regression approach • • Rpart [Breiman93] A H hybrid approach • • SVR [Drucker97] State-of-the art forecasting methods • XGBoost [Chen16] Data set • Evaluation measures: 150 seasonal time series from various sources • Rank ∈ {1, 2, 3, 4,5,6,7,8} • 100 splits (100 train, 50 test); from train data • Accuracy degradation compared to the best method set 10,000 new time series are created An Automated Forecasting Framework based on Method Recommendation for Seasonal Time Series 12 André Bauer Motivation Related Work Approach Evaluation Conclusion
Evaluation Degradation S* S L S B A C A R A H Mean 1.000 1.409 1.249 1.235 1.172 1.159 Median 1.000 1.045 1.076 1.016 1.035 1.032 SD 0.000 3.674 0.427 2.458 1.382 0.382 MAPE A C A R A H ETS tBATS sARIMA Mean 24.40 23.26 23.68 56.96 36.28 28.12 Median 12.31 13.07 13.18 14.47 10.83 13.00 SD 50.31 40.41 38.52 136.22 98.68 64.72 In terms of recommendation, the hybrid version has the lowest mean and standard deviation Our approach has a lower mean forecast error and standard deviation than the forecasting methods An Automated Forecasting Framework based on Method Recommendation for Seasonal Time Series 13 André Bauer Motivation Related Work Approach Evaluation Conclusion
Autonomic Forecasting Method Selection Takeaways Forecasting is an important task for many decision making fields, for instance, Cloud Computing There is no globally best forecasting method (“ No-Free-Lunch Theorem ”) Free Lunch We propose an automated forecasting approach based on decomposition and method recommendation Our experimental results show that… • Our recommendation approaches perform almost equally • The whole forecasting approach outperforms existing state-of-the-art forecasting methods An Automated Forecasting Framework based on Method Recommendation for Seasonal Time Series 14 André Bauer Motivation Related Work Approach Evaluation Conclusion
Recommend
More recommend
