Analytical Support for Rapid Initial Assessment Charles Twardy, Ed - PowerPoint PPT Presentation

Analytical Support for Rapid Initial Assessment Charles Twardy, Ed Wright, Kathryn Laskey, Tod Levitt, Kellen Leister, Andy Loerch George Mason University C 4 I Center 1


Topics • Rapid Initiative Assessment (IA) challenges • Overview of Mason’s IA methodology • Example 2
 2


Challenge: Analysis Support for Initial IA We focus here: Models can also be rapid initial reused here and beyond. assessment. 3
 3


Solution: Analysis Support for Initial IA • Rigorous, rapid, consistent, re-usable analytic justification for JIEDDO initiative assessments • Fulfills critical need as warfighter requirements grow while budgets tighten and scrutiny increases 4


Rapid Initial IA Requirements • Provide rapid assessments (days to weeks) • Model dependence of relevant Measures of Effectiveness (MOEs) on system & environmental variables • Use available knowledge • Identify information collection priorities • Be consistent, repeatable, & extensible 5/19/10


Initiative Assessment Process 1. _ 2. _ 3. Implement as Probabilistic Model Bayesian network (BN) model for EOD robot 1. Identify MOEs Example MOEs : Casualties per Incident Time to Complete Mission 4. Exercise Model & Analyze Results Weapons Intelligence Gathered 2. Generate Explanation Partial Explanation Example : 5. Determine Sensitive Parameters If there is an IED detonation during robot neutralization, Blue soldiers are not 6. Report Results exposed. The robot may be damaged or destroyed. 6


IA Approach Benefits • Identify relevant MOEs. • Consistent framework for assessing initiatives. • Clearly communicates to decision • Generate an Explanation of how the makers, the assessed impact, initiative is expected to affect MOEs. potential tradeoffs, and the mechanism by which it works. • Implement the explanation as a • Makes the explanation structured, probabilistic model. explicit, executable, and reusable. • Execute & analyze model to assess • Perform what-if, try scenarios, test performance understanding, perform sensitivity analysis. • Determine the “sensitive • Enable development of more parameters” (SPs) to help prioritize informative test plans. information collection. 7


Explanation  Probabilistic Model • Generate explanation of how initiative affects MOE – Clutter can interfere with the ability of the sensor to detect IEDs and cause false positives • Implement explanation as Bayesian network (BN) – Structured, explicit, executable, and reusable – Models how initiative is likely to perform in operation – Supports what-if and sensitivity analysis CPT for Sensor_Result 5/19/10
 8


Available Knowledge • SMEs (at JIEDDO and elsewhere) • JUONS and other needs statements • Initiative documentation • Current suite of equipment & capabilities • Additional contractor knowledge • Blue and Red TTPs • Previous initiatives • Previous models • Previous tests 9
 9


Model Development Spiral New Initiative OFFLINE: Enhance models Develop Model 1st New Iteration Yes Repository Add Class Model ? No Select model From Add Repository Recursive Spiral n th Iteration Prototyping Model Modify Model (n+1) th No Yes Time Model Avail Iteration Sufficient? ? Model Yes Performance No Assessment & Model Sensitive Analysis Parameters 10


Analysis Try Scenarios in the Model, and examine the Calculate individual link strengths: effect on the MOEs For each MOE, find the most influential variables: Intel. Potential Vary some parameters over their range: redDetonatesRobot redDetonation probDisableSuccess robotProbEffective robotReadiness 5/19/10
 11

Specific Technologies RECCE I VOSS on Mast Cougar 6x6 Platform Remote Wpn Sys LNS Gyrocam (VOSS) Remote Wpn Sys EOD Robot Comms EOD Robot Duke v1 In the Remote Deployment System RECCE II Adds LNS Photo from the (S) ATEC C&L Report, July 2008 Duke (v2) 12


Example 1 EOD Robot New Initiative New Initiative OFFLINE: Assess EOD Robot Enhance models Develop 1st Develop 1st - New Class? Yes Iteration Iteration Model Model Model Yes Yes Repository New New Add Class Class Select MOEs ? ? Build 1 st Iteration Model No Select model From Add Repository Recursive Spiral n th Iteration Prototyping Model Modify Model (n+1) th No Yes Model Time Iteration Sufficient? Avail? Model Yes Performance No Assessment and Model Sensitive Analysis Parameters 13


MOEs by Tenet * Tenet Potential MOEs Predict (Intell. Gathered)… Mission Time Prevent Number and relative proportion of each type of IED tactic, Number or (Cost) percentage of interceptions, raids, captures before emplacement, #IEDs/ mission mile Detect-Air P(detect), False Alarm Rate, Sweep Width, Rate of Advance Detect- P(detect), False Alarm Rate, Sweep Width, Rate of Advance, P(spot) Ground Neutralize P(neutralize), Neutralize Time, Intelligence Gathered Mitigate Casualties/Attack, KIA/Attack, WIA/Attack, Damage/Attack Some MOEs suggested by Perry et al., Minimizing the Threat from Improvised Explosive Devices in Iraq , RAND 2007 Some from the RECCE II Initiative Evaluation Plan (AMSAA, August 2008) * Tenet: JIEDDO divided initiatives into “tenets” which roughly follow the “left of boom” timeline. 14


Identification of MOEs Assumptions The EOD robot provides a capability to remotely neutralize (disable or detonate) an IED. If the robot is not available or not successful, a soldier will neutralize the IED. MOE Assumptions and Considerations Time Robot may take longer than an EOD soldier If the robot is unsuccessful, we still must use a soldier P(neutralize Distinguish disable from destroy by robot ) • Replace with generalized, qualitative P(damage) Casualties or • If Red detonates the IED during robot neutralization, soldiers are not Damage per Attack exposed. The robot may be damaged or lost. • If the robot is unavailable, or fails, then a soldier will be at risk. • If the IED is not spotted, robot has no effect on damage / casualties. • If Blue disables the IED, it can be examined for forensic intelligence. P(collecting • If Blue detonates it, there may be some intelligence collected before valuable the detonation. Intelligence) • If Red detonates it, there is little intelligence gained. 15


Implement Explanation as BN Model assumes IED is present and successfully detected. • If robot is available and working correctly, it can be used to attempt to disable or detonate an IED. • If there is a Red detonation during neutralization, Blue soldiers are not 1 exposed. The robot may be damaged or destroyed. • If the robot is not available or not successful, a soldier will be at risk while disabling the IED. • If the robot succeeds in disabling the 2 IED, we can gather forensic intel. • Little intelligence can be collected if the robot detonates the IED. 2 3 1 • Using the robot may take longer than 3 using an EOD soldier. • If unsuccessful, a soldier must still disable the IED. 16


Example 1 EOD Robot (2) New Initiative New Initiative OFFLINE: Assess EOD Robot Enhance models Develop 1st Develop 1st - New Class? Yes Iteration Iteration Model Model Model Model Yes Yes Repository Repository New New Add Add Class Class Build 1st Iteration ? ? Model No Add 1st Iteration Select model Select model Model to Repository From From Add Repository Repository Recursive Time Available? No Spiral n th Iteration n th Iteration Prototyping Model Model Modify Run the Model, Analysis Model (n+1) th No No Yes Model Model Time Time Iteration Sufficient? Sufficient? Avail? Avail? Model Yes Performance Performance No No Assessment and Assessment and Model Model Sensitive Sensitive Analysis Analysis Parameters Parameters 17


Robot Analysis 1: View Effects If the robot is not available … a soldier will be at risk while disabling the IED. If a robot is available and it is working correctly, it can be used to attempt to remotely disable or detonate an IED. Lower risk to soldier, more time If the robot succeeds in disabling the IED, it can be examined for forensic intelligence. Less intelligence can be collected if the robot detonates the IED. 18


Robot Analysis 2: Sensitive Parameters by MOE EOD Robot: Top 5 Sensitive Parameters by MOE. • Assuming robotAvailable, and excluding deterministic functions ClearTime Intelligence Damage redDetonatesRobot redDetonatesRobot redDetonation redDetonation redDetonation redDetonatesRobot robotReadiness probDisableSuccess robotProbEffective robotProbEffective robotProbEffective robotReadiness probDisableSuccess robotReadiness -- Next Steps (as time allows): • Investigate sensitive parameters in more detail • Extend / refine the model: additional variables, situations; extend or refine the state space of important variables; refine local probability distributions. • Identify knowledge requirements for the sensitive parameters • Seek additional information: SMEs, system documents, data collection, … 19