Forecasting with Judgment Forecasting with Judgment Simone - PowerPoint PPT Presentation

Forecasting with Judgment Forecasting with Judgment Simone Manganelli DG-Research European Central Bank (Frankfurt am Main, Germany) Disclaimer: The views expressed in this paper are our own and do not necessarily reflect the views of the ECB or the Eurosystem

Classical Theory of Forecasting Classical Theory of Forecasting • Forecasts serve to make decisions about the future. • Forecast errors impose costs on the decision-maker. • Agents want to minimize expected loss associated to forecast errors. • Classical forecast is the minimizer of the sample equivalent of the expected loss. • See, e.g., Haavelmo (1944), Granger and Newbold (1986), Granger and Machina (2005).

Optimal Point Forecast Optimal Point Forecast Simplest case: • i.i.d. normally distributed observations with variance 1 • quadratic loss function T { y } Want to forecast y T +1 t t 1 = 2 min Q ( ) E [( y ) ] � � � � 0 T 1 + � * 2 E [ y ] 0 E [ y ] FOC: � � � = � = T 1 T 1 + + T ˆ 1 � T y � = � = T t t 1

The Problem The Problem ˆ is the minimiser of: � T ˆ T 1 2 � min Q ( ) T [( y ) ] � = � � � � T t t 1 � T 2 1 2 2 � min { E [( y ) ] T [( y ) ] E [( y ) ]} � � � + � � � � � T 1 t T 1 + + t 1 = � ( � ) � � T 2 min { E [( y ) ] ( )} � � + � � T 1 T + �

Outline Outline Introducing Judgment Risk Analysis of New Estimator Example 1: Asset Allocation Example 2: Forecasting US GDP Relationship with Bayesian Conclusion

Introducing Judgment Introducing Judgment Risk Analysis of New Estimator Example 1: Asset Allocation Start from a Subjective Guess Example 2: Forecasting US GDP Start from a Subjective Guess Relationship with Bayesian Conclusion 2 min Q ( ) E [( y ) ] � � � � 0 T 1 + � Q ( ) E [ y ] 0 � � � � � = 0 T 1 + The sample equivalent of the FOC is: ~ ~ ~ ˆ T 1 ˆ Q ( ) T [ y ] y � � = � � � � � = � � T t T t 1 ~ where � is a subjective guess of the decision maker. NOTE : This is a random variable, which may be different from zero just because of statistical error.

Introducing Judgment Introducing Judgment Risk Analysis of New Estimator Example 1: Asset Allocation Example 2: Forecasting US GDP Relationship with Bayesian Then Do Hypothesis Testing Then Do Hypothesis Testing Conclusion ~ H : is the true mean 0 � ~ ˆ y T ~ N ( 0 , 1 / T ) � � • Choose a confidence level α • η α /2 is the critical value If null is rejected: → First derivatives statistically ≠ 0 → Can be confident to decrease the true objective function ˆ → Only up to the point where new H 0 cannot be rejected * � T

Introducing Judgment Introducing Judgment Risk Analysis of New Estimator Graphical Illustration Graphical Illustration Example 1: Asset Allocation Example 2: Forecasting US GDP Relationship with Bayesian Conclusion Distribution of FOC under H 0 ~ ˆ � Q ( ) � T / 2 � 0 ~ ˆ ˆ ˆ � * Q � ( ) Q ( ) � � T T T

Intuition Intuition Introducing Judgment Introducing Judgment Risk Analysis of New Estimator Example 1: Asset Allocation Example 2: Forecasting US GDP Relationship with Bayesian Conclusion 2 2 E [( y ) ] ( y ) � � � � 1 2 ( y ) � � 1 100*(1- α )% confidence interval ˆ ˆ � � � � + � 1 / 2 1 / 2 � � ~ ˆ � y � = � 1 1 First derivatives not statistically different from zero

Introducing Judgment Introducing Judgment Risk Analysis of New Estimator Example 1: Asset Allocation Example 2: Forecasting US GDP Relationship with Bayesian Conclusion New estimator: ~ ˆ ˆ y if y � � � � � > � T / 2 T / 2 � � � ~ ~ � * ˆ if | y | � = � � � < � � T T / 2 � ~ � ˆ ˆ y if y + � � � < � � � T / 2 T / 2 � � �

Introducing Judgment Introducing Judgment Risk Analysis of New Estimator Example 1: Asset Allocation Example 2: Forecasting US GDP Relationship with Bayesian Interpretation Interpretation Conclusion • α is the probability of committing type I errors, i.e. of rejecting the null when it is true. • Choose low α when confident in subjective guess or if the cost of type I errors is high. • Classical paradigm sets α =1: – Always FOC equal to zero; – No room for subjective guess; – It commits type I errors with probability 1.

Introducing Judgment Introducing Judgment Risk Analysis of New Estimator Example 1: Asset Allocation Example 2: Forecasting US GDP Relationship with Bayesian Problems with Pretest Estimators Problems with Pretest Estimators Conclusion Test the following null hypothesis, for given confidence level α : ~ ˆ H : � = � 0 T Then: • If do not reject, keep the subjective guess • If reject, take the maximum likelihood estimator This is wrong!

Introducing Judgment Introducing Judgment Risk Analysis of New Estimator Example 1: Asset Allocation Example 2: Forecasting US GDP Relationship with Bayesian Conclusion 2 2 E [( y ) ] ( y ) � � � � 1 2 ( y ) � � 1 (1- α )% confidence interval ˆ ˆ � � � � + � 1 / 2 1 / 2 � � ~ ˆ � y � = � 1 1 First derivatives not statistically different from zero

Introducing Judgment Risk Analysis of New Estimator Risk Analysis of New Estimator Example 1: Asset Allocation Example 2: Forecasting US GDP Relationship with Bayesian Conclusion

Introducing Judgment Risk Analysis of New Estimator Risk Analysis of New Estimator Example 1: Asset Allocation Example 2: Forecasting US GDP Relationship with Bayesian Conclusion 0 Let y t ~ i . i . d . N ( , 1 ) � Estimate the mean, given a single observation y 1 (Magnus 02) Compare the risk properties of two estimators: 1) Subjective classical estimator (coincides with Magnus 02) ~ y if y � � � � � > � 1 / 2 1 / 2 � � � ~ ~ � * if | y | � = � � � < � � 1 1 / 2 � ~ � y if y + � � � < � � � � 1 / 2 1 / 2 � � 2) Pretest estimator ~ ~ if | y | � � � � < � � ˆ 1 / 2 P � � = ~ � y if | y | � � � > � � 1 1 / 2 �

Introducing Judgment Risk Associated to f f ( ( y y ) ) Risk Associated to Risk Analysis of New Estimator Risk Analysis of New Estimator Example 1: Asset Allocation Example 2: Forecasting US GDP Relationship with Bayesian Conclusion ~ 0 2 E [( f ( y ) ) ] , 0 � � � = 0 � 4 3 2 1 0 0 0.5 1 1.5 2 2.5 3 3.5 4 OLS (_=1) Subjective Guess (_=0) Pretest (_=0.10) Subjective Classical (_=0.10)

Introducing Judgment Risk Analysis of New Estimator Risk Analysis of New Estimator Example 1: Asset Allocation Example 2: Forecasting US GDP Relationship with Bayesian Monte Carlo Simulation Monte Carlo Simulation Conclusion • Random draws from standard normal • Sample sizes = 5, 20, 60, 120, 240, 1000 • Quadratic loss function • Two estimators: classical, new ( α =0.10) • Evaluated expected loss with MC simulation • Repeat 5000 times and average

Introducing Judgment Risk Analysis of New Estimator Risk Analysis of New Estimator Example 1: Asset Allocation Example 2: Forecasting US GDP Relationship with Bayesian Conclusion 0.7 0.6 0.5 Mean 0 0.4 0.05 0.3 0.1 0.5 0.2 1 0.1 0 5 20 60 120 240 1000

Introducing Judgment Risk Analysis of New Estimator Risk Analysis of New Estimator Example 1: Asset Allocation Example 2: Forecasting US GDP Relationship with Bayesian Implications Implications Conclusion • Good guess (gut feelings) are as important as good econometric models. • Organization of forecasting process: – Subjective guess based on maximum likelihood estimates can never be rejected by construction; – Clear separation b/w: Who provides guess, based on judgment; Who tests the guess, based on econometric models. • Shared responsibility for the quality of the forecasts: – High confidence in bad judgment results in bad forecast.

Introducing Judgment Risk Analysis of New Estimator Example 1: Asset Allocation Example 1: Asset Allocation Example 2: Forecasting US GDP Relationship with Bayesian Conclusion

Introducing Judgment Risk Analysis of New Estimator Example 1: Asset Allocation Example 1: Asset Allocation Generalization Example 2: Forecasting US GDP Generalization Relationship with Bayesian Conclusion d ˆ 0 T U ( ) N ( 0 , ) � T � � � � ~ ˆ * ( ) ( 1 ) [ 0 , 1 ] � � � � � + � � � � � T T * 0 Under the null H : ( ) � � = � 0 T ˆ ˆ ˆ * ' * 1 * ˆ z ( ( )) T U ( ( )) � U ( ( )) � � � � � � � � � � � � T T T T T T � � ˆ * ˆ if z ( ( 0 )) 0 � � � � � = T T , k � ˆ ˆ * * ˆ if z ( ( 0 )) arg max U ( ( )) � > � � � = � � T T , k T T � [ 0 , 1 ] � � * ˆ s.t. z ( ( )) � � = � T T , k �