DECISION MAKING WITH INCOMPLETE INFORMATION MARC C. CANELLAS PH.D. - PowerPoint PPT Presentation

DECISION MAKING WITH INCOMPLETE INFORMATION MARC C. CANELLAS PH.D. DISSERTATION DEFENSE SCHOOL OF AEROSPACE ENGINEERING ADVISOR: DR. KAREN FEIGH , AEROSPACE ENGINEERING, GEORGIA TECH COMMITTEE: DR. BRIAN GERMAN , AEROSPACE ENGINEERING, GEORGIA TECH DR. AMY PRITCHETT , AEROSPACE ENGINEERING, GEORGIA TECH DR. STEVE CROSS , INDUSTRIAL & SYSTEMS ENGINEERING, GEORGIA TECH DR. JUAN ROGERS , PUBLIC POLICY, GEORGIA TECH

PEOPLE ARE CONSISTENTLY REQUIRED TO MAKE DECISIONS TO ACHIEVE MISSION SUCCESS REGARDLESS OF THE TIME PRESSURE AND QUANTITY AND QUALITY OF INFORMATION • Necessary information often not available, 2 or too costly (in time, money) to gather or process 1 • Probabilities, relative importance of cues, and cue values are often unreliable 3 • People tend to use simple rules like heuristics without gathering and processing all information (due to time, cost, experience, etc.) 1,4,5,6 1 Orasanu and Connolly, 1993; 2 Green and Mehr, 1997; 3 Tversky and Kahneman, 1974; 4 Katsikopoulos and Fasolo, 2006; 5 Elwyn et al., 2001; 6 General result of naturalistic decision making and fast-and- 2 frugal research programs

COMMAND AND CONTROL IN THESE DEGRADED AND DENIED INFORMATION ENVIRONMENTS IS A FUNDAMENTAL ISSUE IN MODERN MILITARY OPERATIONS Modern warfighters are becoming technology and information dependent. Need to prioritize information for Need for new tactical training and defense and offense (e.g., radar decision support tools. 1 jamming, cybersecurity attacks) 2 1 Lt. Kessler, US Navy, 2010 2 Conversation with Lt. Cmdr. Beris, MORS METSM 2016 3

TRANSLATING C2D2E INTO DECISION THEORY FOR MODELING AND SIMULATION Command and Control within Degraded and Denied Information Environments (C2D2E) Representing Representing fundamental task decision makers attributes Decision tasks with incomplete Range of decision making strategies information • Experts tend to use Represents unavailable information and naturalistic decision time pressure • making or heuristics Well-studied as a driver of decision making (fewer, more important strategy selection and performance cues) 2 • Has clear mapping to interface design: can alter saliency and accessibility of Novices tend to use information 1 normative and analytic strategies (more cues but less important ones) but sometimes also resort to heuristics when stressed 2 1 Feigh et al., 2012; 2 Garcia-Retamero and Dhami, 2009; Gigerenzer and Gaissmaier, 2011 4

DECISION TASKS FOR THIS THESIS Judgment Decision Making (Classifying single object) (Choice among options) Do the cue values change during the No decision making process? Yes Static Dynamic Yes No Are options presented one at a time? Sequential Single Is all information processed Yes No by the strategy? Analytics Heuristics Decision making for this thesis will examine how analytic and heuristic strategies perform in static, single decision tasks with incomplete information. 5

EXEMPLAR DECISION TASK: SINGLE, STATIC, DECISION MAKING WITH INCOMPLETE INFORMATION Naval Defense Task – Operator of Phalanx CIWS • Radar guided 20mm M61 Vulcan cannon gatling gun capable of firing 4,500 rounds per minute • Last line of defense for every surface combat ship in the US Navy • Can be controlled by a human-operator Operator’s Decision Attributes 1 Decision Task Engage the most dangerous of the known simultaneous targets Single, static decision Attributes of targets are altitude, speed, distance, and size of object 3-5 cues, max for 15 sec Sensors degraded by environment density, countermeasures, etc. Incomplete information 15 seconds from effective horizon (seeing the target) to ship impact Heuristic strategies Altitude Speed Distance Size 0.2 m 2 Target A 5,000 ft 1,200 kts ? 5 nmi ? 0.2 m 2 Target B 1,500 ft 1,500 kts 7 nmi ? ? 6 1 Prengaman et al., 2001

CURRENT FOUNDATIONS FOR DECISION SUPPORT FOR INCOMPLETE INFORMATION ENVIRONMENTS Probabilistic Evidence Consumer and Marketing Behavioral Decision Making Acquisition Research + + + + Mathematical, computational, + How the information is + Probabilistic frameworks for and human-subjects studies how decision makers or distributed and presented of both analytic and heuristic decision support systems to a decision maker does decision making 1 matter 3 should acquire information 4 - Only measure of incomplete - Only small scale human- - Require assessments of information is total probabilities that are subjects studies and not information 2 untenable in real-world focused on time-stressed environments 5 decision making There is a need for a comprehensive investigation of incomplete information (mathematical, computational, and human-subjects studies) with a focus on providing decision support without requiring often-unreliable information about probabilities, etc. 1 Gigerenzer et al., 1999; Todd, et al., 2011; 2 Martignon and Hoffrage, 2002; Garcia-Retamero and Reiskamp, 2008; 3 Slovic and MacPhillamy, 1974; Kivetz and Simonson, 2000; 4 Nelson, 2005; 7 5 Katsikopoulos and Fasolo, 2006

MOTIVATION, RESEARCH GAPS, AND RESEARCH QUESTIONS Need for new tactical training and decision support Need to prioritize information for defensive and tools for environments with degraded and denied offensive capabilities environments Incomplete understanding of the effect of Analytic techniques rely on information that is distributions of incomplete information on decision often not available or reliable. making performance. 2. Can information acquisition and restriction 1. Does the distribution of incomplete information methods be defined which do not rely on affect decision making performance? If so, probabilities, etc.? When are they effective? when and how? 8

ORGANIZATION OF THE REST OF THIS DEFENSE PRESENTATION 1. Does the distribution of incomplete information 2. Can information acquisition and restriction affect decision making performance? If so, methods be defined which do not rely on when and how? probabilities, etc.? When are they effective? Characterize the Characterize the impact Define heuristic Define new measures of of distribution of effectiveness of the rules Define new measures of Define new measures of information acquisition incomplete information for decision making incomplete information information on decision incomplete information and restriction rules making strategies strategies 9

DECISION MAKING ACCURACY IS DETERMINED BY THE ENVIRONMENT, STRATEGIES, & INCOMPLETE INFORMATION Environmental Parameters (types of, and relationships between, cues and options) Can be affected by training and decision support tools. Accuracy Strategies (Percent of correct decisions (information processing and integration) made) Incomplete Information (distribution of missing information) Measures of Accuracy Characteristics of the Environment Characteristics of Strategies 10

RANGE OF DECISION MAKING STRATEGIES Analytic: Heuristic: Gather as much information as possible to use in a Uses only a subset of “ necessary ” information (generally linear model (generally uses continuous data) uses discretized data into “ good/bad ” categories) Weighted Additive Equal Weighting Tallying 2 Take-Two 3,4 Take-the-Best (TTB) 5 (WADD) 1 (EW) 1 Choose the first Choose the first Linear model with Linear model without Linear model option to discriminate option to discriminate weights weights counting “good” cues with two “good” cues with one “good” cue Multi-attribute Criminal sentencing Consumer choice 11 ; When weights cannot People often search decision making 6 ; decisions 9 ; be determined or for a second, Expert decision Novice decision Medical decision agreed upon 8 confirming attribute 3,4 makers 7 makers 7 making 10 1 Payne et al., 1990; 2 Gigerenzer and Gaissmaier, 2011; 3 Karelaia, 2006; 4 Dieckmann and Rieskamp, 2007; 5 Gigerenzer and Goldstein, 1996; 6 Park, 2004; 7 Garcia-Retamero and Dhami, 2009; 8 Dawes, 1979; 9 von Helversen and Rieskamp, 2009; 10 Kattah et al., 2009; 11 Kohli and Jeddi 11

DEFINING NEW MEASURES OF DISTRIBUTIONS OF INCOMPLETE INFORMATION Task A Task B Altitude Speed Distance Size Altitude Speed Distance Size Target 1 ? ? ? ? Target 1 K K ? ? Target 2 K K K K Target 2 K K ? ? Measure Definition Task A Task B Amount of cue Total information 1 Total information is 4 or 50% Total information is 4 or 50% values known Difference in Option Imbalance is 4 or 100%, Option Imbalance is 0 or 0%, Option Imbalance amount of cue i.e., an operator is familiar with i.e., an operator has equal (Canellas, Feigh, and Chua, values known Target 2 and unfamiliar with amounts of information for 2014, IEEE CogSIMA) between options Target 1. both options. Number of cues Cue Balance is 2 or 50%: Cue balance Cue Balance is 0 or 0%: which have cue An operator has equal (Canellas, Feigh, and Chua, For no cue does the operator values known for information about both options 2014, IEEE SMC) have comparable information. all options for 2 out of 4 cues. 1 Martignon and Hoffrage, 2002; Garcia-Retamero and Rieskamp, 2008) 12