Air Travel Forecast Problem Objectives • Introduction to forecasting methods • Experience with Delphi • Experience with consensus-seeking techniques • Strength/weaknesses of various methods 1 Air Travel Forecast Problem
Methodology Tree for Forecasting Knowledge source Statistical Judgmental Univariate Multivariate Others Self Data- Theory- based Unstructured Structured Role No role based Extrapolation Data models Unaided Role playing Intentions/ mining (Simulated expectations judgment interaction) Quantitative Neural analogies nets Conjoint analysis Rule-based forecasting Feedback No feedback Linear Classification Prediction Game Decom- Structured Judgmental Expert Causal Delphi Segmentation theory position markets analogies bootstrapping systems models Methodology Tree for Forecasting 2 Air Travel Forecast Problem forecastingpriciples.com JSA-KCG September 2005
Techniques for Forecasting Form groups of about 5 to 7 people, then use the: Minutes Delphi procedure 12 First estimate – individual and anonymous 3 Statistical summary – group 20 Group discussion (use consensus technique) 2 Second estimate – individual and anonymous 3 Statistical summary - group 40 3 Air Travel Forecast Problem
Group Results Accuracy Group Rankings : 1 2 3 4 5 Average (Round 2) ranks Judgment Bootstrapping Segmentation Causal model Extrapolations 4 Air Travel Forecast Problem
Discussion Discuss Delphi Expected results When to use Actual Results Initial hypotheses Results in Air Travel study Calculation of your error score Conclusions 5 Air Travel Forecast Problem
Delphi Agreement among experts Your results More agreement among panelists on Round 1 _____ No differences (Round 1 vs. 2) _____ More agreement on Round 2 _____ Findings from literature: Typically more agreement on later rounds Expected accuracy: Which do you expect to be closest to actual ranks? Your opinions Round 1 more accurate _____ Round 2 more accurate _____ No difference _____ Delphi improves accuracy vs. traditional meetings given some expertise among panelists 6 Air Travel Forecast Problem
Round 2: Previous Rankings vs. Your Rankings Average Ranking MBA Adv. Mgmt. You Method (21 groups)* (28 groups)* Judgment 2.2 2.4 Bootstrapping 3.2 2.9 Segmentation 2.2 2.0 Causal 2.6 2.9 Extrapolation 4.7 4.8 *Groups from U.S., Sweden, Norway, and Netherlands 7 Air Travel Forecast Problem
Evidence-based Findings (“>” means “more accurate than”) 1. Objective methods > subjective: especially for large changes 2. Causal methods > naïve: especially for large changes 3. Bootstrapping > Judgment 4. Structured meetings > unstructured 8 Air Travel Forecast Problem
Using the Selection Tree Sufficient objective data ? Judgmental methods Quantitative methods No Yes Good Large changes knowledge of expected relationships No Yes No Yes Conflict among a few Type of Policy analysis Large changes decision makers data likely No Yes No Yes No Yes Cross-section Time series Accuracy Policy Similar feedback analysis cases exist Good Policy Yes No domain Yes No No Yes analysis knowledge Unaided No Yes Type of Yes No judgment knowledge Domain Self Role playing Delphi/ Judgmental Extrapolation/ Causal Conjoint Intentions/ Structured Quantitative Expert Rule-based (Simulated Prediction bootstrapping/ Neural nets/ models/ interaction/ analysis expectations analogies analogies systems forecasting markets Decomposition Data mining Segmentation Game theory) Several methods provide No Yes useful forecasts Selection Tree for Forecasting Methods Single Combine forecastingprinciples.com method forecasts JSA-KCG Omitted January 2006 information? Use unadjusted No Yes Use adjusted 9 forecast forecast
Rankings based on Evidence-based Findings Method Rank Why? Causal model 1.5 Objective and causal Segmentation 1.5 Extrapolation 3 Objective and naïve Bootstrapping 4 Objective/subjective and causal Judgment 5 Subjective and causal Evidence summarized in Armstrong (1985), Long-Range Forecasting, and Armstrong (2001), Principles of Forecasting – see forecastingprinciples.com 10 Air Travel Forecast Problem
Accuracy of the Different Methods of Forecasting U.S. Air Travel, 1963-1968 (Successive updating used) Forecast Horizon Mean Absolute Percentage Error* Years (Number Extrapolation Judgment Econometric Ahead of Forecasts) 1 (6) 5.7 6.8 4.2 2 (5) 12.7 15.6 6.8 3 (4) 17.4 25.1 7.3 4 (3) 22.5 34.1 9.8 5 (2) 27.5 42.1 6.2 6 (1) 29.9 45.0** 0.7 Averages (21) 19.3 28.1 5.8 * The forecasts were lower than actual in nearly all cases. ** Estimated Source: Armstrong & Grohman (1972) in full text at forecastingprinciples.com 11 Air Travel Forecast Problem
Average Error Scores* Round 2 MBAs 7.4 Advanced Mgt. 7.5 Forecasting Experts 8.4 You *Key : Best possible = 0 No information (all ties) = 6 Worst possible = 12 12 Air Travel Forecast Problem
General Advice • Beware of unaided judgment • Be conservative when uncertain – thus, use equal ranks given uncertainty about most accurate method 13 Air Travel Forecast Problem
Recommend
More recommend
