Advanced Visualization for Operational Assessment Sonia Alvidrez 1 , - PowerPoint PPT Presentation

Advanced Visualization for Operational Assessment Sonia Alvidrez 1 , Christopher R. Hale 1 , Richard H. Loreaux 1 and Donald Monk 2 1: SAIC 2: Air Force Research Laboratory, Human Effectiveness Division 1

Structure of the Talk • Introduction • Cognitive Systems Engineering • Systems Engineering Analysis • Visualization Interface Concept Development • Concept Evaluation • Conclusions and the Way Ahead 2

Operational Effects Assessment Visualization Tool (OEAVT) Benefits to Warfighter Objective PH 2 PR 1 PH 2 PR 1 PH 2 PR 1 PH 2 PR 1 PH 2 PR 1 PH 2 PR 1 PH 2 PR 1 PH 2 PR 1 CA: Gain/Maintain air and space superiority over JOA, CA: Gain/Maintain air and space superiority over JOA, CA: Gain/Maintain air and space superiority over JOA, CA: Gain/Maintain air and space superiority over JOA, CA: Gain/Maintain air and space superiority over JOA, CA: Gain/Maintain air and space superiority over JOA, CA: Gain/Maintain air and space superiority over JOA, CA: Gain/Maintain air and space superiority over JOA, PP-2A PP-2A as required, to execute operations as required, to execute operations PP-2A PP-2A PP-2A PP-2A as required, to execute operations as required, to execute operations PP-2A PP-2A as required, to execute operations as required, to execute operations as required, to execute operations as required, to execute operations 100 100 100 100 100 100 100 100 50 50 50 50 50 50 50 50 • Decision-quality knowledge Decision-quality support to to the commander 0 0 0 0 0 0 0 0 1 1 2 2 3 4 5 6 7 3 4 5 6 7 1 1 2 2 3 4 5 6 7 3 4 5 6 7 1 1 2 2 3 4 5 6 7 3 4 5 6 7 1 1 2 2 3 4 5 6 7 3 4 5 6 7 assessment team in defining SI 1 SI 1 SI 1 SI 1 SI 1 SI 1 SI 1 SI 1 SI 2 SI 2 SI 2 SI 2 SI 2 SI 2 SI 2 SI 2 SI 3 SI 3 SI 3 SI 3 SI 3 SI 3 SI 3 SI 3 • Continuous operational critical indicators, managing assessment tied to assessment data, determining objectives operational effectiveness, • Sensemaking of battlespace visualizing/understanding effects complexity and uncertainty. • Uncertainty management • nth-order causal Approach understanding • Faster decision times • Analysis of assessment domain • ID assessment functions Technologies • ID requirements • Design/Refine-Prototype • Information Visualization • Build • Knowledge Management • Operational test • Intelligent Interface Agents • Advanced Search/Data Mining Evidence Accrual Operational Assessment Prediction & Forecasting Indicator Mgmt 3

Effects-based Assessment: Operational Issues • All operations have complex effects. – Desirable and undesirable effects – 1 st , 2 nd and 3 rd order effects – Time delays introduce uncertainty and risk • Understanding the implications of operations is a multidimensional problem. – Effects can be strategic, operational, physical, psychological, … – System of systems is a major conceptualization & modeling challenge 4

Effects-based Assessment: Human Performance Issues • Data m anagem ent – What to measure, when to measure, how often to measure? • Dynam ic assessm ent – Finding appropriate indicators and measures – Integrating results in real time: A data aggregation problem, an interpretation problem. • Plan troubleshooting – How does one decide when to “stay the course” or to recommend changes the strategic plan? – How to evaluate the efficacy of potential changes? 5

Cognitive System Engineering • EBA Domain analysis and cognitive system engineering • System engineering analysis and requirements • Visualization concept development • Develop prototype 6

Concept Mapping: Basic Information • Five “view s” W ork m anagem ent Cognition • Som e overlap Products Collaboration Autom ation 7

Concept Mapping Identified the following cognitive work for EBA: Pre-execution Execution Post-execution • Assessm ent planning • Accrue evidence • I nter-division feedback • Determ ine adversary • Analyze ops results capabilities & likely • BDA COA • Execution tracking • Develop JAOP • Functional dam age • Develop STTM assessm ent • OAT m gm t of EBA • I ntegrate m ission • Predict ops assessm ent effectiveness • Mission assessm ent • PDA • Target system assessm ent 8

Decision Requirements Tables • Identify and characterize assessment decisions that will drive visualization requirements – Task • Critical cues • Critical decisions • Common errors • Actions • Tools used • Collaboration and communication • Data used • Requirements 9

DRT Example • Recognize actionable changes in ongoing air ops • Assess feasibility of plan changes Critical Cues Critical Actions Common Tools Used Communicates Data Used Requirements Decisions Errors with * Change in * Determine * Monitor activity * TBMCS * ISRD * MISREPS * The system pathways when changes trends in areas shall allow and * Plans team * INTSUMS have been indicated by aid in * Change in * DISUMS made in critical cues recognizing weather ongoing air actionable * Combat * Monitor * Etc. ops changes in assessment indicators ongoing air against ops predictions and time • WOE * Determine if * Make * IWPC * ISRD * TPFID * The system resources recommendation shall * Plans team * ATO • Time * TBMCS available for determine if * Predict • Guidance * Combat Ops * Guidance corrective resources are intended and actions available for • Resource unintended corrective profiles * Determine effects of actions what to changes change & * The system * Infer 1 st , 2 nd , amount of shall 3 rd order effects change determine associated with amount of * Determine potential change to WOE for each changes plan, and what objective to change * Determine priorities 10

CORE Systems Analysis • CORE: Commercially available CASE tool – Allows management of the whole project – A wide range of information – DES supports tradeoff & what-if analysis – DoDAF compatibility • What information we put into it – All concept map and CDA information – Other information specified by SMEs • Vetted with SME input • 108 functions 11

CORE Diagram Examples Execution Can be used to generate SV-4 descriptions Make Analyze operational results Accrue evidence recom m endations and data Activity analysis Analyze tactical Determ ine effectiveness assessm ent sum m aries of air cam paign 12

Visualization Interface Concept Development Scenario Operational execution Element Operational Indicator Prediction and Evidence Reporting Assessment Operational Management Forecasting Accrual Requirement Cascading Plan Intelligent Data mgmt “Live” Capability/Support effects troubleshooting Causal queries briefing Requirement Data Temporal Uncertainty link capability mining assessment visualization analysis Functional Requirement Concept 13

14

Concept Evaluation • Purpose: Demonstrate visualization concepts for an integrated ops assessment system – Feedback on concept – Feedback on implementation 15

Conclusions • This methodology enables a meaningful integration of cognitive systems analysis with accepted system engineering and technology development practice. • The cognitive and perceptual work involved in EBA can be captured by a limited, manageable number of hierarchically structured functions. • Visualization technologies must be both broad and deep for success in an EBA domain. 16

The Way Ahead: Potential Long-term Direction Developm ent Focus Potential Technology Solutions PMESII visualization - Hierarchical task network with recursive task blocks - 3-D rendering - Fisheye view on demand Intelligent queries - Queries for spatial, temporal and probabilistic content - Intelligent, automated data acquisition Causal link analysis - Influence nets • between actions and effects - Colored Petri Nets • cause – effect latencies; latencies in - Hybrid dynamical systems observing effects - Temporal causal graphs • temporal effects - Causal graphs & event calculus - Timed failure propagation graphs N th -order effects -Temporal causal graphs - Causal graphs & event calculus 17

18 Questions