Experimental Evaluation of a Joint Cognitive System for 4D Trajectory Management The Third SESAR Innovation Days Rolf Klomp 1 Challenge the future 1
C-SHARE consortium C-SHARE Max Mulder (coordinator) Martijn Mooij Dennis Nieuwenhuisen René van Paassen Annemiek van Drunen Pim van Leeuwen Rolf Klomp Sorin Iacob Tanja Bos Gesa Praetorius Vincent Clot Anneloes Maij 2 Challenge the future 2
Towards higher levels of automation A key pillar of the SESAR Master Plan: Allocate a central role for the human, but supported by a high degree of automation However, the role of the human is not well defined yet ATM network demand Technological advances 3 Challenge the future 3
How can we effectively cope with complex situations in 4D trajectory planning? observed aircraft metering fix 4 Challenge the future 4
By taking a ‘leap of faith’ and always trusting automation? observed aircraft metering fix 5 Challenge the future 5
The challenge of human-systems integration Human-machine interaction Mismatch? sensing perception settings action 6 Challenge the future 6
C-SHARE aims to make automation more transparent observed aircraft metering fix 7 Challenge the future 7
The foundation for effective human-machine coordination is based on a common ground of the problem to solve functional view (FAV) environment functional model (FAR) 8 Challenge the future 8
Applying the multi-disciplinary CSE and EID paradigms is an iterative process Solution Space Diagram Discrete route-based info Initial Travel Space implementation Vector field method HMI designs Final JCS experiment version 9 Challenge the future 9
Breakdown of the Travel Space Representation travel space observed aircraft metering fix Δψ +10kts +20kts +30kts V MAX Challenge the future 10 10
Breakdown of the Travel Space Representation safe field of travel observed aircraft restricted field of travel metering fix Challenge the future 11 11
The travel space describes the constraints for problem solving rather than a single optimised solution safe field of travel observed aircraft restricted field of travel metering fix Challenge the future 12 12
Allowing for automation and the human operator to work together as team players automated advisory observed aircraft metering fix Challenge the future 13 13
Example use of the Travel Space Representation Challenge the future 14 14
The Travel Space is an egocentric representation ? ? observed aircraft metering fix ? ? Challenge the future 15 15
Research has resulted in the development of a software- based Joint Cognitive System (JCS) Challenge the future 16 16
C-SHARE second human-in-the-loop experiment Aim of the study: • To investigate the effectiveness of the Travel Space Representation under varying airspace and traffic conditions Independent variables: • Traffic orderliness: structured, unstructured • Scale of perturbations: small, medium, large Dependent Measures: • Loss of separation and overall performance • Workload and acceptance • Use of automated advisories Challenge the future 17 17
Experiment set-up Two sub-experiments: • Manual Control task • Automated advisories Participants: • (2x) 12 Participants • 3 groups: ATCos, domain experts, students Procedure: • 45 minute briefing and training • 6 experiment runs of 6 minutes (4x speed) • 2 hours total Challenge the future 18 18
Results Loss of separation and overall performance: • An expected breakdown in human-automation coordination during the more complex scenarios did not occur during either sub-experiment • In both experiments 2 loss of separation events occurred over of 2232 controlled flights • Similarly 2 Restricted Area intrusions occurred Challenge the future 19 19
ISA workload self-assessment • In both experiments the relative workload increased significantly with perturbation scale, however, never became unmanageable • In sub-experiment 2 workload significantly decreased for unstructured traffic, likely due to the availability of automated advisories Challenge the future 20 20
CARS acceptance rating • Acceptance was highest for students, followed by domain experts and finally ATCos • Acceptance did not significantly decrease in more complex scenarios Challenge the future 21 21
Automated advisories • Use of advisories was highest for students, followed by domain experts and finally ATCos • More advisories were used in complex scenarios • The number of advisory rejections shows that the Travel Space enables the operator to understand, veto, and re-direct automation Challenge the future 22 22
Issues in human factors and system performance measurements • Currently no methods to evaluate joint human-automation systems • Methodologies focus heavily on the distinction between the human and the system parts • Results are biased by experience and are highly dependent of individual strategies • There is no clear baseline of a future operational ATM concept Challenge the future 23 23
Overall project outcome • Proven that C-SHARE approach to the design of decision support tools can greatly benefit human-automation coordination • One of the first working (scalable, multi-platform) joint human-automation prototypes for trajectory-based operations • Opportunities for further research have been identified Challenge the future 24 24
Future research Bridging the gap in the Abstraction Decomposition Space components whole system knowledge Functional Purpose Abstract Functions Generalized Flow based management C-SHARE Functions Physical Functions Physical Form technology Challenge the future 25 25
Questions? Contact: R.E.Klomp@TUDelft.nl Challenge the future 26 26
Experimental Evaluation of a Joint Cognitive System for 4D Trajectory Management The Third SESAR Innovation Days Rolf Klomp Challenge the future 27 27
There is a risk of introducing breakdowns in human-automation coordination • “Out-of-the-loop” situation awareness • Vigilance problems • Transient workload peaks • Skill degradation • Overreliance • Lack of trust • … Challenge the future 28 28
Example displays using the ecological approach Challenge the future 29 29
The multidisciplinary approach is based upon the proven CSE and EID frameworks The five steps of CSE: • Work Domain Analysis what is the purpose of the system? • Control Task Analysis what must be done? • Strategies Analysis how can it be done? • Cooperation Analysis how can work be shared? • Worker Competencies how can operators best be supported? Challenge the future 30 30
The Abstraction Hierarchy shows the opportunities, constraints and relationships of the problem domain overall system goals underlying laws and principles realization of abstract functions Challenge the future 31 31
The multidisciplinary approach is based upon the proven CSE and EID frameworks EID: making visible the invisible AUTOMATION Challenge the future 32 32
Iterative approach scope definition and task evaluation design evaluation D1.1 concept FAR conceptual evaluation D4.4 D2.2 JCS version 1 HITL 1 D3.2 D2.3 JCS version 2 HITL 2 D3.3 workshop with stakeholders Challenge the future 33 33
C-SHARE first human-in-the-loop experiment Aim of the study: • To validate the Travel Space concept and obtain expert feedback in an early design stage • Held with 3 former ATCOs involved in SESAR activities at the EEC in Brétigny Challenge the future 34 34
C-SHARE final workshop • 16 international domain experts and future end-users with a broad range of expertise • Hands-on demonstration of the JCS • SWOT analysis to identify strengths, weaknesses, opportunities and threats to guide future research activities demonstration cubicle Interactive SWOT Challenge the future 35 35
Recommend
More recommend
