Managing Model Risk A Practitioner Perspective Douglas Gardner - PowerPoint PPT Presentation

Managing Model Risk – A Practitioner Perspective Douglas Gardner October 5, 2011 0

Wells Fargo’s Approach  Wells Fargo has a “hybrid” approach to model risk management, relying on multiple teams  Each line of business (LOB) or corporate function is responsible for managing all its risks, including model risk which involves both model governance (e.g., model inventory, risk ranking, documentation, approvals, issue tracking) and technical validation. Model developers are responsible for practicing sound model development procedures and documentation.  Corporate Risk (CR) sets policies, oversees LOB model risk governance and validation, and acts as a validation center of excellence for LOBs that choose to outsource validation.  Internal Audit reviews compliance with policy at both the LOB and CR levels. It now includes model SMEs in order to effectively review technical model requirements. For LOB reviews, there is close cooperation with CR regarding policy requirements and interpretation.  Model risk committees at both the corporate and LOB level are employed to oversee model risk management.  Modeling seminars are used to disseminate modeling knowledge and model risk management best practices. 1

Assessment of OCC 2011-12  OCC 2011-12 is the culmination of previous regulatory guidance, regulatory capital requirements , industry best practices and previous examiner feedback – although a significant enhancement and framed somewhat differently , nothing new came as a large surprise.  Challenge for most institutions likely lies with the details outlined in the new guidance and the emphasis on several topics that have not received as much attention previously  Board of Directors’ responsibilities  Assessing aggregate model risk  Detailed guidance around model development  Detailed guidance around ongoing monitoring of models  Vendor guidelines for model validation  Continuous nature of process  “Model risk is too important to be left just to quants ” 2

Regulatory Expectations for Managing Model Risk Mark Levonian Mark Levonian Office of the Comptroller of the Currency October 2011

New regulatory guidance New regulatory guidance • OCC Bulletin 2011 ‐ 12 (April 2011) Supervisory Guidance on Model Risk Management p y g • Replaces OCC Bulletin 2000 ‐ 16 Replaces OCC Bulletin 2000 16 • Developed jointly with Federal Reserve Developed jointly with Federal Reserve • Available at www occ treas gov • Available at www.occ.treas.gov Levonian ‐ OCC ‐ October 2011 2

Risk should be managed Risk should be managed • Using models within any business brings potential for adverse consequences from p q decisions based on models that are incorrect or misused or misused • Managing any risk requires: • Managing any risk requires: – Identifying the sources of risk – Assessing the risk – Mitigating or controlling the risk g g g Levonian ‐ OCC ‐ October 2011 3

Some principles Some principles • Effective governance and controls needed Effective governance and controls needed • Consider materiality • Model development process matters • Validation a core element but not the only • Validation a core element, but not the only element – Is model performing as expected given design objectives and business use? – Validation depends on context of use • “Effective challenge” is key Effective challenge is key Levonian ‐ OCC ‐ October 2011 4

What makes challenge effective? What makes “challenge” effective? Compensation structure Incentives Functional independence p Organizational culture T Technical training h i l t i i Effective Competence Critical thinking Challenge Practical experience Stature within organization Influence Influence Reporting lines Reporting lines Management support Levonian ‐ OCC ‐ October 2011 5

Governance and controls Governance and controls • Model risk management (including validation) is a process, not a one ‐ time event p , • Need for strong policies, procedures, testing • Roles and responsibilities must be clear l d ibili i b l – Ownership, controls, and compliance • Documentation is part of managing • Boards of directors and internal audit have a B d f di t d i t l dit h role to play Levonian ‐ OCC ‐ October 2011 6

www.occ.treas.gov/news ‐ issuances/bulletins/2011/bulletin ‐ 2011 ‐ 12.html Levonian ‐ OCC ‐ October 2011 7

Mark Levonian Mark Levonian Office of the Comptroller of the Currency Mark Le onian@occ treas go Mark.Levonian@occ.treas.gov Levonian ‐ OCC ‐ October 2011 8

OCC 2011-12 Guidance on Model Risk Management Creating a Comprehensive Model Risk Management Framework October 5, 2011

Roles and Responsibilities MODEL VALIDATION/ MODEL OWNER MODEL RISK Ensure models are properly developed, Responsible for overall Model Risk documented, tested, implemented and used. Program. Ensure models have undergone appropriate Independent validation with focus on validation and approval processes. “effective challenge”. Provide all necessary information for Maintain Model Inventory, Issues validation activities. database, and provide reporting on Address model problems as they rise. Model Risk to Sr. Management and BoD. SR. MANAGEMENT AND BoD Set bank’s model risk appetite. INTERNAL AUDIT Responsible for ensuring a strong model Assess overall effectiveness of and risk management framework is in place. compliance with model risk management framework for individual models and in Reviews periodic model risk reporting. aggregate. 1

Areas of Focus to Implement Guidance  Create overall governance structure – Model approvals – Issues escalation – Reporting on model risk  Model Development Documentation and Testing  Model Implementation Documentation and Testing  Model Change Management Process – Change in model structure or assumptions – Change in use 2

Areas of Focus to Implement Guidance  Ongoing model monitoring – Performance monitoring – Risks and limitations, external conditions – Process risks- data and systems  Model Validation – Conceptual soundness – Vendor models – Outcomes analysis- backtesting, benchmarking, stress testing, sensitivity testing  Policies and Procedures 3

Key Challenges  Ensuring everyone understands and accepts their role in model risk management  Staffing  Education  Quantification of Model Risk 4