Air Force Research Laboratory Trust in Automation Research: Current - PowerPoint PPT Presentation

Air Force Research Laboratory Trust in Automation Research: Current Directions and Gaps Date: 5 APR 17 Joseph Lyons, PhD Technical Advisor Human Trust & Interaction Branch Air Force Research Laboratory Integrity  Service  Excellence DISTRIBUTION STATEMENT A: Approved for Public Release 1

Outline • Personal Background • Trust Background – Definition & Measures – Trust Literature • Auto-GCAS Project – What is it? – Survey results – Interview results – Lessons Learned • Future Directions & Gaps – Shared awareness and shared intent DISTRIBUTION STATEMENT A: Approved for Public Release 2

Personal Background • Ph.D. I/O Psych, Minor HF Psych - Wright State University 2005 • Senior Research Psychologist for Air Force Research Laboratory – since 2005 • Research Interests: human-machine trust, interpersonal trust, leadership, Org Science • 2011-2013 served as Program Officer for Air Force Office of Scientific Research – Chartered the Trust & Influence Portfolio – Air Force lead for Minerva • 2013 Returned to WPAFB – started the Human Insight and Trust (HIT) Team – Dedicated to understanding human-machine trust for AF domains – 9 Civilians 1-2 Military, 1 post-doc, approx. 6 FTE contractors DISTRIBUTION STATEMENT A: Approved for Public Release 3

HIT Video DISTRIBUTION STATEMENT A: Approved for Public Release 4

HIT Background • Prior/Current HIT Research – Automated versus human aids (Lyons & Stokes, 2012) – Transparency and trust (Lyons, 2013; Lyons & Havig, 2014; Lyons et al. 2016a) – Trust and suspicion processes (Lyons et al., 2011; Bobko et al., 2014) – Trust of software code (Alarcon et al., in press; Hodgkin et al., 2016) – Impact of different error types on trust (Guznov et al. 2015; 2016) – Human-Robot Interaction • Signaling intent – Dialogue, behavior, and rationale • Social influences (Stokes et al., 2016) • Human-Agent Teaming (Wynne & Lyons, in press) – Pilot trust of landing aids in Degraded Visual Environments – Trust evaluations of fielded systems (Lyons et al., 2016b; Ho et al., in press; Kolitai et al., 2014) DISTRIBUTION STATEMENT A: Approved for Public Release 5

Trust Background • Trust = willingness to be vulnerable w/out the ability or capacity to monitoring (Mayer et al., 1995) • Necessary Conditions – Human intention – Relational – must have a referent (otherwise its dispositional) – Risk – without it trust is obsolete • What to Measure? – Intention to be vulnerable – Trustworthiness • Ability, benevolence, integrity (Mayer et al., 1995) • E.g., Machine characteristics, “Learned trust” (Hoff & Bashir, 2015) – Reliance behavior • Must be volitional, should be in risky context – more than mere cooperation – Dispositional trust – Physiology? DISTRIBUTION STATEMENT A: Approved for Public Release 6

Trust Background… “a small sliver” • Management, e-commerce, Human Factors, Robotics, Social Psych… trust research is pervasive • Mayer et al. (1995) • Lee & See (2004) • Parasuraman & Manzey (2010) • Hancock et al (2011) • Chen & Barnes (2014) • Technology Acceptance Model • Merritt et al. (2016) • Hoff & Bashir (2015) DISTRIBUTION STATEMENT A: Approved for Public Release 7

The Problem – Inaccurate Trust • Why is this so complicated??? – Errors for highly automated systems can be catastrophic (Onnasch et al., 2014) • Asiana flight 214 (2013) – overreliance • Cruise ship Concordia (2012) – under-reliance – Trust is dynamic & driven by multiple factors • Social, Affective, Cognitive, Dispositional… • Absolute values will change with changing context – Importance of predictors may also change – Perceptions of trustworthiness may be inaccurate – Unintended consequences (Parasuraman & Riley, 1997) – Few studies using real operators, real-world technology & real-world consequences (R3) DISTRIBUTION STATEMENT A: Approved for Public Release 8

Automatic Ground Collision Avoidance System • Time-to-collision-based automated system – Components: Collision avoidance Algorithm, Digital Terrain Elevation Data, interface with flight control computer, HUD interface • Behavior: wings-roll-to-level maneuver, 5-G pull up • Objectives: do not interfere, do no harm, avoid collision DISTRIBUTION STATEMENT A: Approved for Public Release 9

AGCAS Video DISTRIBUTION STATEMENT A: Approved for Public Release 10

Why Auto-GCAS? • Why is this a good domain to study trust? – Fielded automated DoD system – high level of automation • Supports AF interests in autonomy & future technologies • Supports pilot safety – Real-life implications for safety • “Most sophisticated AF safety system ever fielded” • Four “saves” to date since 2014 – Trust-rich context • Potential for false alarm/miss, prior system links, imperfect reliability, novelty – Good place to test trust antecedents – ecological validity – Engineer/test concerns • Distrust - Too much trust • Impact of HUD display unknown • Prior warning-based systems distrusted • Acknowledgment of team: Dr. Nhut Ho, Lauren Hoffmann, Garrett Sadler, Capt Eric Fergueson, 1Lt Anna Lee Van Abel, Samantha Cals, Kolina Koltai, Maj Casey Richardson, Mark “Tex” Wilkins, Air Vehicles Directorate, Air Force Flight Test Center DISTRIBUTION STATEMENT A: Approved for Public Release 11

AGCAS Field Study • AF Trust research lacks data on operator trust in fielded systems with high levels of autonomy • Four-year longitudinal study (started FY15) – Examine operational pilot trust evolution in AGCAS overtime – Identify and document user experience, concerns, impact, and benefits of technology as they emerge – Obtain operational pilots feedback on AFRL future automated/autonomous systems • AACAS & Autonomous Wingman • Uses survey and interview methods – Visited 13 bases (6 were repeat visits in year 2) – Year 1: Survey (N = 142), Interview (N = 168) – Year 2: Survey (N = 100), Interview (N = 131) DISTRIBUTION STATEMENT A: Approved for Public Release 12

Survey Measures Rated on 7-point Likert Scales Trust (4 items, α = .95) “I can count on Auto-GCAS to work when needed” Performance (3 items, α = .64) “Auto-GCAS is reliable” Transparency (3 items, α = .84) “I understand how Auto-GCAS works” Benevolence (3 items, α = .74) “I think Auto-GCAS was designed to help me” Benefits (2 items, r = .82) “I benefit from having Auto-GCAS installed on my plane” Confidence (3 items, α = .62) “I feel confident with Auto-GCAS installed on my plane” Aggressive flying (2 items, r = .75) “Auto-GCAS allows me to fly lower to the ground” Automation Schema (Merritt et al., 2016; 3 items, α = .69) “Automated systems rarely make mistakes” DISTRIBUTION STATEMENT A: Approved for Public Release 13

Year 1 Correlations with trust (N = 142) Predictor r Outcome r Schema .29** Confidence .68** Performance .63** Aggressive .14 Transparency .23** Benevolence .43** Benefits .55** Note: ** p < .01. DISTRIBUTION STATEMENT A: Approved for Public Release 14

Year 1 Unique Predictors of Trust (N = 142; Lyons et al., 2016c) β R 2 Predictor .47** Schema .13† Performance .39** Transparency .10 Consistent with Hoff & Bashir, 2015 Benevolence .06 Benefits .25** Note: † p < .10, ** p < .01. DISTRIBUTION STATEMENT A: Approved for Public Release 15

Year 2 Correlations with trust Predictor r Outcome r Schema .23* Confidence .79** Performance .62** Aggressive .18 Transparency .16 Benevolence .24* Benefits .45** Note: * p < .05, ** p < .01 DISTRIBUTION STATEMENT A: Approved for Public Release 16

Year 2 Regression β R 2 Predictor .39** Schema .14 Performance .53** Less uncertainty – diminished impact of Individual differences Transparency .05 Stronger performance perceptions with Benevolence -.05 “Save” stories pervasive Likely ceiling effects for transparency, Benefits .10 Benevolence, and benefits Note: ** p < .01 DISTRIBUTION STATEMENT A: Approved for Public Release 17

Experienced Versus Novice Effects • How does experience influence trust and aggressive flying? – Operationalized as: • Experienced > 500 flt hrs., Novice < 500 flt hrs. – Potential for complacency • Generational differences in automation trust and use – Mostly DoD-irrelevant “older” samples – Focused on cognitive decay – Need research on military-relevant ages DISTRIBUTION STATEMENT A: Approved for Public Release 18

Experienced Versus Novice Effects (Lyons et al., under review) t (92) = 2.37, p < .05. DISTRIBUTION STATEMENT A: Approved for Public Release 19

Key Drivers of Trust via Interview (Ho et al., in press) • System performance – Operational “Save” videos • 4 “saves” since 2014! – Perceived nuisance-free operations – Test rides via Pilot Activated Recovery System • Business case – Need for system – Perceived benevolence via system extremely high • Error attribution for early errors • Transparency – HUD display – Increases SA, indicates the system is working • Pedigree of Test Community in the AF • Activations are handled as learning opportunities DISTRIBUTION STATEMENT A: Approved for Public Release 20