C-18 Model Risk in Financial Systems Casualty Actuarial Society - PowerPoint PPT Presentation

C-18 Model Risk in Financial Systems Casualty Actuarial Society Annual Meeting November 6-9, 2011 Parr Schoolman, FCAS, MAAA, CERA Sr. Managing Director Aon Benfield Analytics

Agenda Physics Envy Section 1 Insurance Risk Checklist and Implications for Model Risk Section 2 Conclusion Section 3 1

Section 1: Physics Envy

Model Risk and Uncertainty Physics Envy  “Physics Envy Can be Hazardous to Your Wealth”, (Lo, Mueller 2010) – The goal : to be able to build models of economic systems that are as predictive as those used in physics – The problem : people are less predictable than electrons – Risk and the Taxonomy of Uncertainty • Level 1: Complete Certainty • Level 2: Risk without Uncertainty • Level 3: Fully Reducible Uncertainty • Level 4: Partially Reducible Uncertainty • Level 5: Irreducible Uncertainty – Sixth Level: Zen Uncertainty…”attempts to understand uncertainty are mere illusions” 3

Risk vs. Uncertainty Physics Envy and Financial Modeling Complete Partially Reducible Fully Reducible Irreducible Risk without Certainty Uncertainty Uncertainty Uncertainty Uncertainty All states of the Randomness Randomness Randomness Uncertainty beyond the system are governed by a governed by an governed by an reach of probabilistic determined known unknown, but unknown probability analysis exactly probability constant probability distribution, with time F=mA distribution distribution varying parameters Known Newtonian Casino Casino with no Model Risk Unknown Unknowns Physics odds posted Black Swans Mathematics, Physics Statistics, Chemistry, Biology Economics Philosophy Source: “Physics Envy Can be Hazardous to Your Wealth”, (Lo, Mueller 2010) 4

Risk vs. Uncertainty Physics Envy and Financial Modeling Complete Partially Reducible Fully Reducible Irreducible Risk without Certainty Uncertainty Uncertainty Uncertainty Uncertainty All states of the Randomness Randomness Randomness Uncertainty beyond the system are governed by a governed by an governed by an reach of probabilistic determined known unknown, but unknown probability analysis exactly probability constant probability distribution, with time F=mA distribution distribution varying parameters Known Newtonian Casino Casino with no Model Risk Unknown Unknowns Physics odds posted Black Swans Mathematics, Physics Economics Philosophy Source: “Physics Envy Can be Hazardous to Your Wealth”, (Lo, Mueller 2010) 5

Risk vs. Uncertainty Physics Envy and Financial Modeling Complete Partially Reducible Fully Reducible Irreducible Risk without Certainty Uncertainty Uncertainty Uncertainty Uncertainty All states of the Randomness Randomness Randomness Uncertainty beyond the system are governed by a governed by an governed by an reach of probabilistic determined known unknown, but unknown probability analysis exactly probability constant probability distribution, with time F=mA distribution distribution varying parameters Known Newtonian Casino Casino with no Model Risk Unknown Unknowns Physics odds posted Black Swans Mathematics, Physics Statistics, Chemistry, Economics Philosophy Biology Source: “Physics Envy Can be Hazardous to Your Wealth”, (Lo, Mueller 2010) 6

Risk vs. Uncertainty Physics Envy and Financial Modeling Complete Partially Reducible Fully Reducible Irreducible Risk without Certainty Uncertainty Uncertainty Uncertainty Uncertainty All states of the Randomness Randomness Randomness Uncertainty beyond the system are governed by a governed by an governed by an reach of probabilistic determined known unknown, but unknown probability analysis exactly probability constant probability distribution, with time F=mA distribution distribution varying parameters Known Newtonian Casino Casino with no Model Risk Unknown Unknowns Physics odds posted Black Swans Mathematics, Physics Statistics, Chemistry, Economics Philosophy Biology Source: “Physics Envy Can be Hazardous to Your Wealth”, (Lo, Mueller 2010) 7

Risk vs. Uncertainty Physics Envy and Financial Modeling Complete Partially Reducible Fully Reducible Irreducible Risk without Certainty Uncertainty Uncertainty Uncertainty Uncertainty All states of the Randomness Randomness Randomness Uncertainty beyond the system are governed by a governed by an governed by an reach of probabilistic determined known unknown, but unknown probability analysis exactly probability constant probability distribution, with time F=mA distribution distribution varying parameters Known Newtonian Casino Casino with no Model Risk Unknown Unknowns Physics odds posted Black Swans Mathematics, Physics Statistics, Chemistry, Economics Philosophy Biology Source: “Physics Envy Can be Hazardous to Your Wealth”, (Lo, Mueller 2010) 8

Risk vs. Uncertainty Physics Envy and Financial Modeling Insurance Risk Space Complete Partially Reducible Fully Reducible Irreducible Risk without Certainty Uncertainty Uncertainty Uncertainty Uncertainty All states of the Randomness Randomness Randomness Uncertainty beyond the system are governed by a governed by an governed by an reach of probabilistic determined known unknown, but unknown probability analysis exactly probability constant probability distribution, with time F=mA distribution distribution varying parameters Known Newtonian Casino Casino with no Model Risk Unknown Unknowns Physics odds posted Black Swans Statistics, Chemistry, Economics Philosophy Mathematics, Physics Biology 9

Insurance Risk Uncertainty Continuum  Key characteristics to consider in the evaluation of insurance risk uncertainty and its impact upon model risk: – Is the key loss driver a physical (earth, wind, fire, …) or behavioral process (lawsuits, market price volatility, regulatory actions…)? – Do we have a structural or statistical model of the underlying process? – How stable are historical trends, correlations ? Is past performance relevant for prospective risk? 10

Section 3: Insurance Risk Checklist & Implications for Model Risk  Asset Risk  Reserve Risk  Catastrophe Risk

Credit Crisis Lessons Asset Risk Modeling Challenges Monthly Change in Corporate Credit Spreads BBB 3-5 Yr 0.5 0.4 0.3 0.2 0.1 Standard Deviation 0 8.4 basis points -0.1 -0.2 -0.3 -0.4 -0.5 Dec-91 Dec-92 Dec-93 Dec-94 Dec-95 Dec-96 Dec-97 Dec-98 Dec-99 Dec-00 Dec-01 Dec-02 Dec-03 Dec-04 Dec-05 Dec-06 LTCM 9/11 Source: Bloomberg, Aon Benfield Analytics Crisis 12

Credit Crisis Lessons Asset Risk Modeling Challenges Monthly Change in Corporate Credit Spreads BBB 3-5 Yr 2 October 2008 1.5 spread change 147.2 basis points 1 17x Standard Deviation result 0.5 0 -0.5 -1 Dec-91 Dec-92 Dec-93 Dec-94 Dec-95 Dec-96 Dec-97 Dec-98 Dec-99 Dec-00 Dec-01 Dec-02 Dec-03 Dec-04 Dec-05 Dec-06 Dec-07 Dec-08 Source: Bloomberg, Aon Benfield Analytics 13

Insurance Model Risk Characteristics Checklist Asset Risk Insurance Model Risk Characteristics Checklist: Asset Risk Physical Behavioral Risk Process Structural Statistical Model of Process Stable Unstable Parameter Stability Partially Reducible Fully Reducible Irreducible Uncertainty Uncertainty Uncertainty  Risk driven by behavioral “animal spirits”  Risk management models (trading VaR models) statistical rather than structural in nature  Underlying trends and correlations unstable – Input assumptions too often confused with model output 14

Reserve Risk  What is Reserve Risk?  Management: How much will my actuary’s reserve estimate change this year?  Actuary: How far off could my estimate of ultimate losses be? Management Actuarial View View SCHEDULE P - PART 2 - SUMMARY 1 2 3 4 5 6 7 8 9 10 11 12 Years in Which INCURRED NET LOSSES AND DEFENSE AND COST CONTAINMENT EXPENSES REPORTED AT YEAR END ($000 OMITTED) DEVELOPMENT Losses One Two Were Incurred 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 Year Year 1 Prior 241.8 262.5 279.0 295.7 308.0 315.5 323.1 326.6 331.3 336.5 5.2 9.9 2 2001 234.1 235.4 237.7 240.7 242.5 243.6 244.3 244.7 244.9 245.0 0.1 0.3 3 2002 XXX 231.4 226.7 228.4 231.3 232.3 232.5 232.5 232.5 232.6 0.1 0.1 4 2003 XXX XXX 244.8 234.3 231.9 230.4 229.3 229.2 228.5 228.1 -0.4 -1.0 5 2004 XXX XXX XXX 259.9 246.1 241.3 237.9 235.3 233.8 232.8 -1.0 -2.4 6 2005 XXX XXX XXX XXX 277.8 267.4 262.5 258.8 256.3 254.8 -1.5 -3.9 7 2006 XXX XXX XXX XXX XXX 262.9 256.3 257.6 248.1 244.1 -4.1 -13.6 8 2007 XXX XXX XXX XXX XXX XXX 283.7 284.5 274.4 272.5 -1.8 -12.0 9 2008 XXX XXX XXX XXX XXX XXX XXX 325.7 322.2 318.8 -3.4 -6.9 10 2009 XXX XXX XXX XXX XXX XXX XXX XXX 291.9 288.2 -3.7 XXX 11 2010 XXX XXX XXX XXX XXX XXX XXX XXX XXX 289.9 XXX XXX 12 Totals XXX XXX XXX XXX XXX XXX XXX XXX XXX XXX -10.5 -29.5 15

Reserve Risk Initial Booked Loss Ratio Volatility – Commercial Auto Liability 14.0% 12.0% 2001 Calendar Year Loss Reserve Development 10.0% 2000 8.0% 2002 6.0% 4.0% 2003 1999 2.0% 2004 1998 0.0% 55% 60% 65% 70% 75% 80% 85% 1996 2008 -2.0% 2005 2009 1997 2006 -4.0% 2007 2010 -6.0% Accident Year Ultimate Loss Ratio at 12 Months Source: SNL US Industry Aggregate Schedule P, Aon Benfield Analytics 16