INVENTORY Lloyds Banking Group Scotiabank TIAA NYU Tandon School - PowerPoint PPT Presentation

INVENTORY Lloyds Banking Group Scotiabank TIAA NYU Tandon School of Financial Risk Engineering

THE MANY CHALLENGES OF MODEL INVENTORY Can Any Financial Firm Truthfully Claim to Have a Complete and Accurate Model Inventory? Jon R. Hill, Ph. D. Professor of Financial Risk Engineering, NYU Independent Consultant in Model Risk Management jh7050@nyu.edu jonhill@optonline.net

ABOUT THE SPEAKER Jon Hill, Ph. D., is a Subject Matter Expert in Model Risk Management and is an independent consultant to financial institutions. Dr. Hill is also an adjunct Professor at New York University where he teaches a graduate course in Model Risk Management and Governance in the Department of Finance and Risk Engineering. Dr. Hill serves as Head of the New York Chapter of the Model Risk Mangers International Association (MRMIA), established in 2018 . The association’s purpose is to promote awareness of model risk to the broader risk and financial communities and to provide a forum for topical discussion of model risk management challenges and regulatory requirements. Jon is a former Managing Director at Credit Suisse with over twenty years of experience in various areas of quantitative finance. As head of the Global Head of Model Risk Standards at Credit Suisse he led a team comprised of 14 model risk managers in New York London, Zurich, Mumbai and Singapore. Jon’s team had responsibility for the ongoing identification, measurement, risk rating, inventory and monitoring of CS corporate model risk across all business units, regions and legal entities. Prior to joining Credit Suisse in 2017, Jon founded and led the Validation team for Market and Operational Risk Models at Morgan Stanley for 6 ½ years. Prior to Morgan Stanley Jon performed hands-on model validations at Solomon Smith-Barney ( which later became Citigroup) was a member of a quantitative finance research team. Dr. Hill Holds a Ph.D. in biophysics and is a published author in the field of model risk management. Jon is a frequent speaker and chairperson at MRM conferences in both the US and Europe.

• Brief Review of SR11- 7: the US Federal Reserve Board’s comprehensive guidance for model risk management Inventory • Requirements for model inventory articulated by SR-11-7 • Is it a Model or a Tool? Session • Assessing and assigning models into risk tiers • Model Risk Management versus Model Development & Ownership Outline • Eleven favorite excuses for ‘Why my model is not really a model.’ • Three of the most daunting challenges for model inventory to be discussed by today’s session panelists

“For the purposes of this document, the term model refers to a In April, 2011 the FRB & quantitative method, system, or approach that applies statistical, economic, financial, or mathematical theories, OCC Jointly Issued SR11- techniques, and assumptions to process input data into 7/OCC2011-12.* quantitative estimates .” “ A model consists of three components: an information input component, which delivers assumptions and data to the model; This 21-page Document a processing component, which transforms inputs into Set the Bar for Model Risk estimates; and a reporting component, which translates the Management (MRM) at estimates into useful business information.” All Conforming Firms * ‘ Supervisory guidance on model risk management ’, 4th April, Supervisory and Regulatory Letter SR11-7. Document may be downloaded from: https://www.federalreserve.gov/supervisionreg/srletters/sr1107.htm

“Banks should consider risk from individual models and in the aggregate. Aggregate model risk is affected by interaction and dependencies among models; reliance on common assumptions, data, or methodologies; and any other factors that could adversely affect several models and their outputs at SR11-7 Established the same time. With an understanding of the source and Requirements for Firm magnitude of model risk in place, the next step is to manage it Wide Aggregation of properly. Model Risk, Posing of A guiding principle for managing model risk is "effective Effective Challenge (by challenge" of models, that is, critical analysis by objective, MRM), and Tiering informed parties who can identify model limitations and assumptions and produce appropriate changes. Models by Risk Rating The range and rigor of validation activities conducted prior to first use of a model should be in line with the potential risk presented by use of the model.

Banks should maintain a comprehensive set of information for models implemented for use, under development for implementation, or recently retired. While each line of business SR11-7 Established A Formal may maintain its own inventory, a specific party should also be Requirement for Firm-Wide charged with maintaining a firm-wide inventory of all models, Model Inventories with which should assist a bank in evaluating its model risk in the Comprehensive Information aggregate. Any variation of a model that warrants a separate validation should be included as a separate model and cross- About Models in Use or referenced with other variations. Recently Retired as well as Their Intended Use and Any The inventory should describe the purpose and products for which the model is designed, actual or expected usage, and any Upstream Data or Model restrictions on use. It is useful for the inventory to list the type Dependencies. and source of inputs used by a given model and underlying components (which may include other models), as well as model outputs and their intended use.

IN SUMMARY • SR11-7 requires banks to create and maintain a complete and accurate inventory of all models. • SR11-7 requires banks to be able to aggregate model risk across the firm, which in turn requires an understanding of inter- dependencies within the firm’s model ecosystem • Virtually every financial firm tries to satisfy these requirements for a rigorous MRM inventory through verbal attestations (voluntary declarations) by model owners and stakeholders (developers, supervisors and users). • Because attestation is a manual and error-prone process it is questionable whether any firm can claim to have a complete and accurate model inventory. • Understanding model inter-dependencies is even more problematic since it relies on multiple levels of attestation

THE CRUX OF THE MATTER: DEVELOPERS AND RISK MANAGERS RESIDE IN DIFFERENT SILOS Model Development vs. Model Risk Management ✓ Model Risk Managers are tasked with identifying and mitigating the holistic model risks that reside in a firm’s model ecosystem fabric. ✓ Model developers are tasked with designing and implementing and testing models that efficiently and accurately convert input data into useful outputs. ✓ These two groups tend to work completely independently within separate silos at most financial firms. ➢ As a result model developers tend to have little interest in modifying their models to accommodate the requirements of MRM.

This practice is partly motivated by a single line in SR11-7 that Provides enough wiggle room for a Potential Escape Clause for Model Owners: Today All Leading “The range and rigor of validation activities conducted prior to first use of a model should be in line with the potential risk Financial Firms Engage in a presented by use of the model.” Process of Assigning Their Model risk tiering allows firms to substantially reduce Models into High, Medium validation requirements for the lower risk model tiers. As a and Low Risk Tiers result, some model owners may look for creative ways to have their software ‘widgets’ bucketed into the lowest risk tier. This is one of the reasons that at most firms a rigorous model discovery process will invariably reveal a set of potential models that have been ‘hiding in the shadows’ .

What Is It Really? True Model? Model Risk Managers at many firms that strive to be SR11-7 compliant have developed procedures to classify all quantitative software implementations into one of three categories: model, non-model (calculator) or something in- between (near-model, tool, etc.). FLOD: would prefer not to have anything they own classified Near- ? as a model to avoid the overhead of a rigorous, SR11-7 Model? compliant full validation. SLOD: has to answer to senior management and regulators about why a software implementation was or was not classified as a model and therefore subject or not subject to the rigors of a full SR11-7 model validation. These opposing priorities sometimes result in ‘rather animated Non-Model? conversations’ between first and second LODs over which cloud domain a particular implementation should be assigned to: true model, near-model or non-model.