Toward Understanding Robust & Collaborative Monitoring - PowerPoint PPT Presentation

Toward Understanding Robust & Collaborative Monitoring (12RH05COR) PI: Dr. Chris Myers (AFRL/RHAC) Co-PI: Dr. Andrew Howes (U. of Birmingham) Co-PI: Dr. Rick Lewis (U. of Michigan) Additional Collaborators: Dr. Joseph Houpt (Wright State Univ.) AFOSR Program Review: Mathematical and Computational Cognition Program Computational and Machine Intelligence Program Robust Decision Making in Human-System Interface Program (Jan 28 – Feb 1, 2013, Washington, DC)

Dynamic Monitoring 2

Dynamic Monitoring 3

Dynamic Monitoring • Visual encoding • Eye movements • Decision making – Is what is encoded a target? – What should be encoded next? • Mouse movements & clicking • Often involves coordinating with another individual 4

List of Project Goals Years 1-2: – Adaptive boundedly optimal model of foveated vision Visual encoding, eye movements, decisions – Adaptive, boundedly optimal model of motor control Mouse movements & clicking Years 2-3: – Integrated models of foveated vision and motor control Model performs monitoring task – Collaborative version of integrated model Coordinating with another individual Human—Model performance comparisons throughout 5

List of Project Goals Years 1-2: – Adaptive boundedly optimal model of foveated vision Visual encoding, eye movements, decisions – Adaptive, boundedly optimal model of motor control Mouse movements & clicking Years 2-3: – Integrated models of foveated vision and motor control Model Performs monitoring task – Collaborative version of integrated model Coordinating with another individual Human—Model performance comparisons throughout 6

Cognitively Bounded Rational Analysis 1. Specify the architecture & environment – Info available to process; – Processing constraints; – Explicit utility function; 2. Specify space of possible task strategies 3. Compute expected subjective utilities 4. Determine strategies with highest payoff & compare to human data 7

Saccadic Selectivity • Distractor ratio task – Find the O 3 color:45 shape 24 color:24 shape 45 color:3 shape – 15 different ratios – Potential for adaptive feature search 8

Saccadic Selectivity *Shen, J., Reingold, E. M., & Pomplun, M. (2000). Distractor ratio • Adaptive behavior* influences patterns of eye movements during search. Perception, 29, 241-250. 9

Adaptive Visibility Model* *Kowler, E. (2011). Eye movements: the past 25 years. Vision Research, 51 (13), 1457-1483 • Cognitively bounded rational searcher • Scaled down task – find the O: 5:1 O O O O O X O X X O O X O O 3:3 X X O X X O X 1:5 • Constraints: – Foveated vision  Limited encoding in periphery – Feature noise  red or green? X or O? – Spatial noise  at which position? 10

Adaptive Visibility Model Computational Cognitive Process model Constraints on perceptual encoding Optimally integrated percepts 11

Adaptive Visibility Model Intelligent | Random Spatial & Feature Noise Bayes Rule Target | No Target 12

Object Percept • Represents the likelihood that the shape and color at each location is/is not the same as the target (O) Fixated Location Truth: X X O O X O O Model’s percept: Color: .4 .6 .3 .1 .4 .3 .2 .2 .4 .4 .5 .5 .6 .7 Shape: 13

Obtain Percept: Spatial Noise • Feature for a location may come from a neighbor X X O O X O O X X O X X O O color shape “Illusory conjunctions” • Fovea  µ = 0; σ = 0.1 Neri & Levi, 2006; Levi, 2008; Poder & Wagemans, 2007 • Parafovea  µ = 0; σ = 10 14

Obtain Percept: Feature Noise • Sampled from normal distribution O O O O O X O • Fovea  µ = 0; σ = 0.3 • Parafovea  µ = 0; σ = 1 15

Optimal Integration • Update posterior probabilities – Across all possible displays – Given sample from current fixation location • Probability that a display contains the target for all displays 16

Decision Variable Calculation • Two decision variables: – Present : sum over posteriors for all displays that contain a target – Absent : sum over posteriors for all displays that DO NOT contain a target • Threshold – Set to 0.85 – Potential for strategic variability 17

Different Strategies • Intelligent: – Sequential – start at a location and move from left to right, and back around, until threshold reached – Posterior Driven – next fixation at location most likely to contain the target • Unintelligent: – Random – uniform; sample location w/replacement 18

Model Results: Random Strategy 19

Model Results: Sequential Strategy 20

Model Results: Posterior Strategy 21

Model Results: Posterior Strategy 22

Model Results: No Spatial Noise Fixations per trial for random strategy Fixations per trial for sequential strategy Fixations per trial for random strategy Fixations per trial for sequential strategy 3.0 ! ! 5.6 ! ! ! ! ! ! ! ! ! 5.4 ! ! ! ! ! ! ! ! ! ! ! 2.4 ! ! ! ! ! ! ! ! ! ! 2.9 5.2 5.4 2.3 Number of Fixations Number of Fixations Number of Fixations Number of Fixations 2.8 5.0 5.2 Target absent Target absent Target absent Target absent Target present Target present Target present Target present 2.2 4.8 2.7 5.0 4.6 2.6 2.1 ! 4.8 ! ! 4.4 ! ! ! ! 2.5 ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 2.0 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 Number Same − Feature1 Distractors Number Same − Feature1 Distractors Number Same − Feature1 Distractors Number Same − Feature1 Distractors Fixations per trial for look − for − targets strategy Fixations per trial for look − for − targets strategy 2.2 ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 3.8 2.1 Number of Fixations 3.6 Number of Fixations Target absent Target absent Target present Target present 2.0 3.4 3.2 1.9 ! ! ! ! ! ! ! ! ! ! ! ! ! ! 3.0 23 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 Number Same − Feature1 Distractors Number Same − Feature1 Distractors

What about salience? • Salience makes similar predictions – But only with explicit IOR – And aid from uncertainty for decision threshold • Hence, really salience+ 24

Differentiating b/t the Models • Find a target (e.g., X): + Initial fixation crosshairs 500 ms; either 8°, XO + OX 12°, 14°, or 16° from + ♠♣ ♥♦ 500 ms Current Trial: $0.005 3000 ms Total Earned: $3.445 25

Differentiating b/t the Models • Find a target (e.g., X): XO OX Salience AVM Prediction Prediction • Parafoveal calibration procedure to help account for individual differences of the bounds in periphery • Results will: • Differentiate between two potential models • Indicate appropriate bounds for monitoring model 26

Motor Control & Integration • 3 models, 2 developed, 1 under development • Experiment to differentiate b/t models • Stay tuned – Had a video teaser, but tough to run from Dayton… 27

Summary • Adaptive visibility model for monitoring – Empirical study to distinguish b/t it and salience • Motor control models for moving and clicking under development – 3 models – more on this next year • Integration of perceptual and motor models – Exploring the use of POMDPs 28

Team Dr. Chris Myers (PI; Air Force Research Laboratory) Dr. Andrew Howes (Co-PI; Univ. of Birmingham, UK) Dr. Rick Lewis (Co-PI; Univ. of Michigan) Dr. Joe Houpt (Asst. Professor; Wright State University) Mr. Joe Benincasa (Research Assistant; UDRI) 29

List of Publications Attributed to the Grant Lewis, R. L., Howes, A., & Singh, S. (submitted). A bounded optimality approach to psychological theory: Linking mechanism and behavior through utility maximization. TopiCS in Cognitive Science Myers, C. W., Lewis, R. L., & Howes, A. (in preparation). Boundedly optimal adaptation during visual search. To be submitted to the 35 th Annual Cognitive Science Conference ; Berlin, Germany. Myers, C. W., Lewis, R. L., & Howes, A. (in perparation). A boundedly optimal model of parafoveal search. To be submitted to the 12 th International Conference on Cognitive Modeling ; Ottawa, Canada. 30

Robust Monitoring (Myers) Technical Approach: Research Objectives: • Empirically investigate robust human • Identify bounds on relevant processes behavior during visual monitoring • Apply bounds to processes • Rigorously test cognitively bounded • Determine optimal behavior given bounds rational analysis as a principled solution • Compare cognitively bounded optimal behavior to empirical data Budget ($k): DoD Benefits: • Understanding of how humans achieve YR 1 YR 2 YR 3 YR 4 robust adaptation will facilitate the design of systems they operate 150k 250k 150k 0k • An understanding of robust human adaptation will help toward the Project Start Date: Oct. 2012 development of training curricula and Project End Date: Oct. 2014 methodology 31