The Illusion of Method: Challenges of Model-Based Safety Assessment Oleg Lisagor Linling Sun Tim Kelly
Overview Why do we trust safety assessment? Why do we trust FTA? Can we trust current MBSA techniques on the same basis? What do we need to do to better justify adequacy of the models and assessment? Some “red herrings” in justification of the MBSA adequacy? Why MBSA techniques often focus on synthesis of Fault Trees and FMEA? ISSC 2010: The Illusion of Method - 2
Safety Assessment as Hypothesis The result of any safety assessment is a hypothesis Regardless of the methods used How we think the system will behave under conditions of failure Cannot be fully “validated” or “proven” Can check for consistency with design models Can check for consistency with equipment test data (FMESs) Can check for consistency with experience with similar systems Can review and, sometimes, “test” Yet safety assessment is routinely trusted Why? ISSC 2010: The Illusion of Method - 3
Making it Real: Fault Trees FTA is a structured methodology – not only notation! Ground Rules and Key Principles for Fault Tree construction “Primary-Secondary-Command” “Immediate and Necessary cause” Provide guidance and keywords Facilitate completeness of assessment FTA is a well-defined and well-publicised methodology Professional scrutiny Training and expertise Historical experience Strengths and limitations are known Standard errors and misconceptions are well-publicised Review of Fault Trees ISSC 2010: The Illusion of Method - 4
Why is MBSA different? FTA is a structured methodology – not only notation! FTA is a well-defined and well-publicised methodology Historical experience Review of Fault Trees ISSC 2010: The Illusion of Method - 5
Why is MBSA different? Methodologies are not well-defined Often focus on notation rather than safety engineering concepts Proliferation of idiosyncratic languages Many techniques – little information on how are they related There are currently three “purist” approaches to MBSA Focus on what to model rather than how to assess the system No public guidance Little historical evidence (understandably) What systems are inappropriate for the application of the technique? What are the “error inducing features”? Does it actually work? Can not justify any confidence “by construction” ISSC 2010: The Illusion of Method - 6
MBSA: Model Review Component-wise review is inadequate Adequacy of component models is context dependent Context of component models is not obvious Unlike the context of intermediate events in FTs “Emergent behaviour” Not attributable to individual components Simulation can facilitate a model-wide review Exhaustive simulation is infeasible Need strategies for selecting “simulation cases” ISSC 2010: The Illusion of Method - 7
MBSA: Adequacy Arguments Intuitive and/or Implicit approach to justifying confidence in the MBSA is not sustainable Too little experience to trust the “gut feeling” New challenges Too many risks Confidence must be explicitly justified Model adequacy argument Side-by-side with the system safety argument Incorporated into the overall safety case ISSC 2010: The Illusion of Method - 8
MBSA: Adequacy Arguments ISSC 2010: The Illusion of Method - 9
MBSA: Adequacy Arguments Top Level Argument Top Level Argument ConstructionArg Methodology Argument over Model-Based Safety adequacy of Assessment Methodology construction process {T}, that is [claimed to be] and methodology followed in construction of the Model {M} MethodologyAdequacy ProcessAdequacy The modelling Construction process for methodology is adequate model {M} was adequate MethodologyImplementation MethodologyDefinition MethodologyAppropriate Construction process for Model Methodology is robust and The methodology {T} is {M} has adhered to the appropriate for the type of adequately defined methodology {T} system and the type of safety analysis performed Competency ArchitectureElicitation ConceptsDefinition ArchitectureVerification HistoricalArg Safety engineers responsible for Key concepts of the The methodology for verification The methodology for model constructon have received methodology and their Argument over previous of adequacy and correctness of determining model architecture adequate training in modelling relationship are adequately application of the the model architecture is is adequately defined methodology defined methodology and similarity adequately defined of application context ComponentModelling The methodology for definition of (detailed) component models is adequately defined ISSC 2010: The Illusion of Method - 10
MBSA: Adequacy Arguments ISSC 2010: The Illusion of Method - 11
MBSA: Justification of Assumptions Currently the argument is weak Only weak evidence for some goals Doesn’t recognise the crucial role of modelling assumptions A challenge even for the traditional approaches More critical for MBSA Should cite, manage and justify Assumptions log Part of the adequacy argument in the safety case ISSC 2010: The Illusion of Method - 12
MBSA: Adequacy Arguments Top Level Argument Top Level Argument ConstructionArg Methodology Argument over Model-Based Safety adequacy of Assessment Methodology construction process {T}, that is [claimed to be] and methodology followed in construction of the Model {M} MethodologyAdequacy ProcessAdequacy The modelling Construction process for methodology is adequate model {M} was adequate MethodologyImplementation MethodologyDefinition MethodologyAppropriate Construction process for Model Methodology is robust and The methodology {T} is {M} has adhered to the appropriate for the type of adequately defined methodology {T} system and the type of safety analysis performed Competency ArchitectureElicitation ConceptsDefinition ArchitectureVerification HistoricalArg Safety engineers responsible for Key concepts of the The methodology for verification The methodology for model constructon have received methodology and their Argument over previous of adequacy and correctness of determining model architecture adequate training in modelling relationship are adequately application of the the model architecture is is adequately defined methodology defined methodology and similarity adequately defined of application context ComponentModelling The methodology for definition of (detailed) component models is adequately defined ISSC 2010: The Illusion of Method - 13
MBSA: Adequacy Arguments ISSC 2010: The Illusion of Method - 14
MBSA: Justification of Assumptions Currently the argument is weak Only weak evidence for some goals Doesn’t recognise the crucial role of modelling assumptions A challenge even for the traditional approaches More critical for MBSA Should cite, manage and justify Assumptions log Part of the adequacy argument in the safety case ISSC 2010: The Illusion of Method - 15
MBSA Adequacy Illusions “Our safety analysis is based on the design model of the system; provided the system is implemented as designed the model is, by definition, valid.” “Our safety assessment model is expressed in a language with formally defined semantics which ensures that the model is correct by construction.” “Our MBSA technique is based on formal methods which guarantee validity of analysis results.” “Since our MBSA technique allows to synthesise fault trees and since fault trees are “tried-and-tested” there are no new challenges” ISSC 2010: The Illusion of Method - 16
Conclusions Novel MBSA techniques pose new challenges Use of traditional formats hides these challenges To justify use of MBSA evidence some work is necessary What is being modelled: Clear conceptual methods definition How to model: Guidance Comprehensive and public Strategies for model review and selective simulation Industrial Application (alongside traditional methods) Must justify adequacy of the models explicitly In the system safety case Need to recognise importance of the assumptions Identify, manage and justify in the model adequacy argument ISSC 2010: The Illusion of Method - 17
Questions CAN YOU TRUST YOUR MODEL? ISSC 2010: The Illusion of Method - 18
Recommend
More recommend
