Model Risk Management Patrick Ferrell - AVP Nathan Schlindwein – Sr. Auditor IASA April 27, 2017
Overview • Company background • Why the focus? • Challenges • Internal Audit’s role • Implementation plan 2
RLI Profile Specialty Property/Casualty Insurance company serving “niche” or underserved Traded on NYSE (RLI) – Sox compliant Operates primarily in the United States with over 40 locations and more than 950 employees 2016 Financial Status Gross Written Premium of $875M Assets of $2.8B Consistently outperforms industry profitability 3
Products We Offer 4
Underwriting Profit RLI has achieved 21 straight years of a combined ratio* below 100 5
Model Risk - Defined • The possibility of financial loss, incorrect business decision, misstatement of external financial disclosures or damage to the company’s reputation arising from : – Possible errors in the model design – Misapplication of model, or model results, by model users – Errors in data inputs or assumptions – Incomplete processing – Unauthorized changes 6
What is MRM? • Definition : Model risk management formalizes the approach to the design, implementation, use and governance of key models within the business • Should be part of a broader ERM framework and report to high level within company • A robust MRM can mitigate risks and is becoming a vital component of ERM and corporate governance • Disclosure requirements - To date, NAIC is calling for disclosure of model validation within ORSA; no guidance on expectations (leading to range of emerging practices) 7
Why the Focus on Model Risk Management (MRM)? • Emphasis began with the banking industry • During financial crisis, unexpected losses and incorrect management decisions arose because management didn’t understand the intended purpose of the model • North American CRO Council issued a paper in 2012 outlining eight core principles for strong model risk management o Factors increasing the importance of modeling and need to appropriately validate models include: Growth of products requiring complex valuation models Regulator and rating agency expectations Critical models insurers use may not be subject to internal control testing or external audit Range of emerging validation practices 8
Why the Focus at RLI? • Our Audit Committee and executive management began asking what we were doing to mitigate this risk at RLI 9
Challenges for RLI • Ownership • Model definition • Implementation 10
Challenge – Ownership • Model risk management is a cross-company initiative involving multiple departments and potential interdependencies – upstream and downstream processes • Requires a cross-functional coordination with consistent application of model risk ranking and control/documentation requirements • Requires a broad knowledge of all departments and their potential use of models • RLI Solution: Internal Audit facilitates but does not own Model Risk Management; Creation of a (cross-functional) Model Risk Governance Committee comprised of senior-level management 11
MRM for Insurers * Source: PwC, “Insurance Model Risk Management Maturity Framework and Diagnostic Tool”, January 2014 12
Model Risk Governance Committee • Consists of: – President & COO – SVP, Risk Services – VP, CFO – VP, Corp Development (in charge of ERM) – AVP, IAS (ex-officio member) • Responsibilities include: – Approval and ownership of Model Risk Management Policy (along with any changes thereafter) – Approval for any changes to policy document or changes in MRM process as a whole 13
Roles and Responsibilities • Model owner – Works with dept or product VP and responsible for: – Development of inventory of models used in their area – Risk ranking of each model – Documentation and testing of applicable controls on an annual basis • Department or Product VP – in addition to the above, also responsible for : – Annual attestation regarding completeness of model inventory and operating effectiveness of controls around each model’s risk(s) – Reporting inventory and testing results to MRM Facilitator annually 14
Internal Audit’s Role • Model Risk Management (MRM) Facilitator (currently IAS) – responsible for: – Maintenance of policy document – Gathering of model information company-wide and aiding departments in identifying higher-risk models – Facilitating update and attestation process annually – Reporting corporate model risk inventory and results of testing to Model Risk Governance Committee annually – Assist departments in identifying and designing appropriate controls and monitoring procedures 15
Challenge - Model definition • “What is a model?” – Every spreadsheet? – Complex calculations? – Statistical component? • No right answer, but significant impact on resources needed to implement effective MRM program 16
RLI’s definition • A “model” consists of three components: – An input component, which delivers assumptions and data to the processing component – A processing component, which transforms inputs into estimates – A reporting component, which translates estimates into useful business information 17
Model risk characteristics • Key (higher risk) models are defined by the following characteristics: – Are key drivers of important decisions – Involve external communication or reporting (financial reporting, rating agencies, reinsurers, regulators) – Financial statement balances and/or disclosures rely upon the model and the financial statement balances are significant – The model is complex due to nature of algorithm or volume of inputs – The model results are not predictable or cannot benchmarked to another model • Non-key (lower risk) models are identified by the following: – Used for general business decisions and model outputs are not directly recorded or disclosed in f/s – Financial statements or disclosures which rely upon the model are not significant 18
Implementation • Creation and approval of Model Risk Management Policy • Creation of risk ranking and control criteria • Development of model risk ranking template and supplemental documentation worksheet 19
Risk Ranking and Control Criteria • Criteria to be considered when evaluating individual model risks: – Expertise of the user – Expertise of the model creator – Level of automation – Level of change control – External reporting – Likelihood and severity of error • Criteria to consider when establishing and documenting mitigating controls: – Reconciliation – Secondary review – User access control – System edit controls – Independent validation 20
Example: Model Risk Template 21
Example: Model Risk Template 22
Example: Model Risk Template 23
Right-size Risk Weightings • Majority of RLI’s models are owned by the Risk Services Department – Met with owner of Risk Services model to discuss Key and Non-key model risk rankings – Made adjustments to risk weightings based on discussion 24
Example: Risk Ranking Guidance 25
Example: Risk Ranking Guidance 26
Annual Assessment & Attestation Process • Model Risk Management SharePoint site – Maintains Inventory – Tracks Assessment and Attestation 27
Questions?
Recommend
More recommend
