Tibor Bosse, Alexei Sharpanskykh, Jan Treur, Henk Blom, Sybert - PowerPoint PPT Presentation

Agent-Based Modelling of Hazards in ATM Tibor Bosse, Alexei Sharpanskykh, Jan Treur, Henk Blom, Sybert Stroeve SESAR Innovation Days, Braunschweig, Germany, Nov 27-29, 2012 vrije Universiteit amsterdam

Contents 1. MAREA project and motivation 2. A generalised set of hazards in ATM 3. Agent-based modelling of hazards for resilience analysis TOPAZ model constructs – VU model constructs – New model constructs – 4. Analysis of results 5. Conclusions and future research SESAR Innovation Days, Braunschweig, Germany, Nov 27-29, 2012 2

ATM: an Open Socio-Technical System Complexity and performance variability in ATM  Distributed human operators and technical systems  Considerable interconnectivity between the agents  Internal and external uncertainties and disturbances  Human role is important to cope efficiently with uncertainties and disturbances SESAR Innovation Days, Braunschweig, Germany, Nov 27-29, 2012 3

Resilience Engineering “Design of socio -technical systems that are able to resist a wide variety of demands, variations, degradations and disruptions” Human flexibility and system oversight are essential  Away from error-thinking  Towards a broad view on human performance in an overall system SESAR Innovation Days, Braunschweig, Germany, Nov 27-29, 2012 4

Mathematical Approach towards Resilience Engineering in ATM (MAREA) Aim To develop a mathematical modelling and analysis approach that allows to bring Resilience Engineering at work for the complex ATM system Focus on human performance Humans dealing with uncertainties and  non-nominal conditions Psychological and organisational models  SESAR Innovation Days, Braunschweig, Germany, Nov 27-29, 2012 5

Identification of Hazards Hazard = “Anything that may influence safety”  Events / conditions / performance aspects  Humans / systems / environment  Interactions NLR ATM Hazard Database  ATM safety assessments  Hazard brainstorm sessions  4000+ hazards SESAR Innovation Days, Braunschweig, Germany, Nov 27-29, 2012 6

A Set of Generalised Hazards Development Selection of unique hazards (Set I) Validation Generalization of hazards (Set II) 525 4000+ Pilot mixes up ATC clearances Flight plans of ATC system and FMS differ Wrong waypoints in database Pilot validates without checking Weather forecast is wrong Transponder sends wrong call-sign Alert causes attentional tunneling Resolution of conflict leads to other conflicts Track drop on controller HMI Risk of a conflict is underestimated Animals on the runway False alert of an airborne system Controller has wrong SA about intent of aircraft Contingency procedures have not been tested SESAR Innovation Days, Braunschweig, Germany, Nov 27-29, 2012 7

How to Model Hazards for Resilience Analysis? Requirements  Modelling at level of individual humans and technical systems  Possibility to capture complex non-linear dynamics  Availability of computational tools Agent-based modelling  ‘Agent’ = autonomous system interacting with environment  Agents represent behaviour at local level  Behaviour at global level ‘emerges’ in simulations Human4 Human5 Human6 System5 System3 Human1 Human2 System1 Accident Human3 System4 System2 Organizational context SESAR Innovation Days, Braunschweig, Germany, Nov 27-29, 2012 8

Main Research Goal “To increase the percentage of potential hazards modelled by existing accident risk assessment methods for ATM” More specifically:  Model hazards from ‘Set I’ via ABM approaches Three Phases: 1. TOPAZ model constructs (SID 2011) 2. VU model constructs (ATOS 2012) 3. New model constructs (SID 2012) SESAR Innovation Days, Braunschweig, Germany, Nov 27-29, 2012 9

TOPAZ Model Constructs C1 Human Information Processing C8 Human Error C2 Multi-Agent Situation Awareness C9 Decision Making C3 Task Identification C10 System Mode C4 Task Scheduling C11 Dynamic Variability C5 Task Execution C12 Stochastic Variability C6 Cognitive Control Mode C13 Contextual Condition C7 Task Load SESAR Innovation Days, Braunschweig, Germany, Nov 27-29, 2012 10

TOPAZ Model Constructs - Example Cognitive Control Mode (C6) error strategic probability degree tactical of opportunistic control scrambled subjectively available time SESAR Innovation Days, Braunschweig, Germany, Nov 27-29, 2012 11

Matching Model Constructs to Hazards • Informal approach to assess ‘coverage’ of hazards • For each hazard- model combination perform ‘mental simulation’ • Multiple analysts • Example: ‘Pilots do not react to controller call due to high workload’ scrambled call no response unimportant low priority SESAR Innovation Days, Braunschweig, Germany, Nov 27-29, 2012 12

TOPAZ Model Constructs – Hazard Coverage Cultural differences between airlines  ... Controller makes a reading error  Human error  Multi-agent SA Controller is fatigued and sleepy  ... 81 Failure of GPS system Not  System mode Lack of experience in Covered degraded modes  ... Covered Pilot reports wrong position  Human error 155 Partly  Multi-agent SA Procedure change  confusion 30  Multi-agent SA Pilots do not react to controller call  Decision making due to high workload  ...  Task identification  Task scheduling Controller ignores an alert  Cognitive control mode  Multi-agent SA  ... SESAR Innovation Days, Braunschweig, Germany, Nov 27-29, 2012 13

VU Model Constructs MC1 Bottom-up Attention MC7 Trust MC2 Experience-based Decision Making MC8 Formal Organisations MC3 Operator Functional State MC9 Learning MC4 Information Presentation MC10 Goal-oriented Attention MC5 Safety Culture MC11 Extended Mind MC6 Complex Beliefs in Situation Awareness SESAR Innovation Days, Braunschweig, Germany, Nov 27-29, 2012 14

VU Model Constructs - Example Operator Functional State (MC3) External World Operator Actions Processing Provided Effort Task Task Effort Recovery Task Goals Demands Demands Motivation Effort Environment Situational Generated Exhaustion State Aspects Effort Maximal Effort Task Experienced Critical Execution Pressure Point State Basic Expertise Personality Cognitive Profile Profile Abilities SESAR Innovation Days, Braunschweig, Germany, Nov 27-29, 2012 15

VU Model Constructs - Example Trust (MC7) SESAR Innovation Days, Braunschweig, Germany, Nov 27-29, 2012 16

VU Model Constructs – Hazard Coverage A jolly atmosphere on the frequency Complex procedure causes R/T overload  ...  Operator Functional State  Formal Organisation Icing of the wings  ... 36 Controller is fatigued and sleepy Not  Operator Functional State 18 Aircraft picks up beacons Partly with similar frequencies  ... Clutter of audio messages Covered  Information Presentation  Situation Awareness 212 Negotiation problems Pilot-ATC  Trust Controller has low confidence in  ... validity of system alerts  Trust Pilots falling asleep  Operator Functional State  ... SESAR Innovation Days, Braunschweig, Germany, Nov 27-29, 2012 17

New Model Constructs A Unstabilised Approach H Merging or Splitting ATC Sectors B Handling of Inconsistent I Reduced Visibility Information by a Technical System C Sub-optimal Emotional J Weather Forecast Wrong Atmosphere D Complex or Unclear Procedures K Strong Turbulence Leading to Confusion E Changes in Procedures Leading to L Icing Confusion F Human Does Not Know When to M Influence of Many Agents on Take Action Flight Planning G Problems with Access Rights to an N Uncontrolled Aircraft Information System SESAR Innovation Days, Braunschweig, Germany, Nov 27-29, 2012 18

New Model Constructs - Example Sub-optimal Emotional Atmosphere (C) q S q R Emotion  S  SR  R  R SESAR Innovation Days, Braunschweig, Germany, Nov 27-29, 2012 19

New Model Constructs - Example Changes in Procedures Leading to Confusion (E) SESAR Innovation Days, Braunschweig, Germany, Nov 27-29, 2012 20

New Model Constructs – Hazard Coverage Security Intrusion A jolly atmosphere on the frequency  ...  Operator Functional State  Emotion Contagion Unmanned Arial Vehicles 6 16  ... Not Icing of the Wings Partly  Icing Military Aircraft Shoots a Civil Aircraft Down  ... Unstabilised Approach Covered  Approach 244 Standard R/T not adhered to  Confusion Aircraft picks up beacons  ... with similar frequencies  Handling of Inconsistent Info Strong variation in view by a Technical System  Weather  ... SESAR Innovation Days, Braunschweig, Germany, Nov 27-29, 2012 21

Conclusion 38 agent-based model constructs have been identified 13 TOPAZ model constructs • 11 VU model constructs • 14 new model constructs • Result: considerable improvement of hazard coverage 6 16 81 36 Not Partly Not Not 18 Covered Partly TOPAZ + VU + NEW Covered Covered Covered 155 Partly 244 212 30 SESAR Innovation Days, Braunschweig, Germany, Nov 27-29, 2012 22