Levels of Evaluation and Implications for Assessing Training Results - PowerPoint PPT Presentation

Levels of Evaluation and Implications for Assessing Training Results from the Secure LVC Advanced Training Environment Demonstration J E S S I C A D . C O R T E Z , R E S E A R C H E R I / O P S Y C H O L O G I S T W A R F I G H T E R R E A D I N E S S R E S E A R C H D I V I S I O N , 2 9 A p r i l 2 0 2 0 Distribution A. Approved for public release; distribution unlimited. Case Number 88ABW-2018-4376

What is SLATE? SLATE (Secure LVC [Live-Virtual-Constructive] Advanced Training Environment) IS A 40-MONTH ADVANCED TECHNOLOGY DEMONSTRATION • Commanders of Air Force Materiel Command and Air Combat Command co-commissioned • F-35 is Target Program • Air Force Research Laboratory chartered to “vet the Critical Enabling Technologies” required for LVC (“6 Pillars of SLATE”) • Directed by: 711 th Human Performance Wing, Airman Systems Directorate, Warfighter Readiness Research Division • F-15Es chosen as modified live players • Aircraft with LVC-capable Operational Flight Program (OFP) • F-16s as unmodified live players • No change to aircraft OFP • During mission, carry SLATE pod, demonstrating encrypted Air Combat Maneuvering Instrumentation • U.S. Navy joined to augment modified live with F-18s as robust cross-platform players and additional pods Case Number 88ABW-2018-4376

SLATE VIDEO “Training realism shall be maximized through use of the live training domain supplemented by integrated virtual and constructive capabilities.” --DoD Directive 1322.18 Video PA Cleared: 88ABW-2018-5681 3

SLATE ATD Charter “THE 6 PILLARS OF SLATE” THROUGH ALL 3 PHASES Validate LVC infrastructure, interface control and data 1 standards in a government owned, non-proprietary architecture Document training 6 Demonstrate Multiple Independent Levels of Security 2 (MILS), encryption technology and rule sets effectiveness benefits and Evaluate alternative large throughput, MILS capable 3 opportunities with encryptor, radio, data-link and 5G-ATW waveform integrated LVC capabilities 4 Demonstrate Pod form factor 5 Develop select weapon system OFP changes with vendors based upon clear guidelines, resulting in direct risk reduction and leave behind for Program of Record Every effort being made for exportability across infrastructure Case Number 88ABW-2018-4376

SLATE ATD: PILLAR SIX Document Training Effectiveness Benefits and Opportunities With Integrated LVC Capabilities Pillar Six Slides PA Cleared / Case Number 88ABW-2018-5740

Kirkpatrick’s Level 1 “reaction evaluation” measures whether aircrew are satisfied, motivated and interested in the relevance and applicability of LVC as a training and readiness construct. 6 Case Number 88ABW-2019-3130

SLATE ATD: Pillar Six Pillar Six Slides PA Cleared / Case Number 88ABW-2018-5740

SLATE Phases – Pillar 6 • Phase I : • Respondents: 8 • Entity Counts: 147 LVC weapons events • F-15E ML, F-16 and F/A-18 sims • LVC-Enabled Sorties Flown: 12 • Phase II : • Respondents: 11 • Entity Counts: 160 LVC weapons events • F-15E ML, F-16 UL, F-16 sims • LVC-Enabled Sorties Flown: 14 • Phase III : • Respondents: 35 • Entity Counts: 335 LVC weapons events • F-15E ML, F/A-18E/F/G ML, F-16 and F/A-18 sims • LVC-Enabled Sorties Flown: 71 Photos by RHA Pillar Six Slides PA Cleared / Case Number 88ABW-2018-5740

SLATE Focus Point: Training Realism RESULTS IMPLICATIONS THROUGH ALL 3 PHASES Participants’ Feedback Illustrates Aircrew: Participant Responses: Positives • Look for realism that provides instant and “Overall LVC integration. The ability to have 3 types of entities • correct feedback on the network gives multiple opportunities and fighting against ‘realistic’ threats.” • Value a technology-rich innovative • “Flying in formation with virtual and constructive bros. ‘Seeing’ learning environment them in your sensors and being visual, although positions were • Noted the careful planning of scenarios not accurate at times, is game changing. Allows the opportunity to train as part of a larger force package.” By providing powerful new ways to engage aircrew Participant Responses: Things to Work On in operational and strategic-level missions will • “The single frequency war made communications nearly increase their capacity to operate effectively and impossible. Severely degraded tactical operations and the thrive under any environmental condition. ability to focus on the fight.” • “The Link-16 & SLATE pictures are not correlating in ALL cases which causes confusion & possible poor tactical decisions.” Participant responses are a representative subset of a fuller set of comments Pillar Six Slides PA Cleared / Case Number 88ABW-2018-5740

SLATE Focus Point: System Capabilities RESULTS IMPLICATIONS THROUGH ALL 3 PHASES Participants’ Feedback Indicated: Participant Responses: Positives • The scope and application of interactions within "Mort cue. Real-time feedback helps highlight our mistakes or • the system must evolve in order to increase the validates our tactics and helps the learning sink in.“ • “Seeing fake entities on S/A radar worked fairly well seemed system’s value more stable worked as they should with symbology separating • The resources available to meet the challenges of what was real and fake." tomorrow and indicated they would welcome an extensive model library Participant Responses: Things to Work On The system must meet the demands of a changing • “Having constructive aircraft outside airspace does not help environment in order to increase the desirability of tactical training with heavy civilian traffic because we do not the LVC enterprise architecture. To increase its fight aircraft outside the airspace. This needs some heavy value, the system must evolve. admin discussion.” • “More in-aircraft troubleshooting capabilities or instruction.” Participant responses are a representative subset of a fuller set of comments Pillar Six Slides PA Cleared / Case Number 88ABW-2018-5740

SLATE Focus Point: Future Capabilities RESULTS IMPLICATIONS THROUGH ALL 3 PHASES Participants Noted: Participant Responses: Positives • The knowledge management system’s ability to "LFE amounts of red air that can't be replicated at home station.“ • focus on roles and missions, tactics and • "Getting multiple threats to appear without generating all the aircraft needed was amazing.“ situational awareness • “8v8 line with another~8 constructive threats- can't fight that • The system’s ability to determine performance on at home.” measures divided into quality domains A firm grasp on the broad array of factors influencing Participant Responses: Things to Work On aircrews’ performance across process, quality and • "The ADCP software required to run SLATE appears to result in efficiency measures translates into actionable ideas abnormalities in normal avionics operations. Degraded the or practical innovations. capability to focus on tactics by increasing aircrew workload and troubleshooting.“ • “Pods are too user intensive requiring more cross check time to ensure proper operation precluding full up tactical maneuvering.” Participant responses are a representative subset of a fuller set of comments Pillar Six Slides PA Cleared / Case Number 88ABW-2018-5740

SLATE Focus Point: User Satisfaction RESULTS IMPLICATIONS THROUGH ALL 3 PHASES Participants’ Feedback on Analysis of the System: Participant Responses: Positives • Focused on examining actions and procedures "Getting multiple threats appear without generating all • that provide realism and produce more highly the [Live] aircraft needed was amazing.“ • "Nice use of 3-D playback for debrief. Outstanding debrief skilled aircrew tool for surface to air threat engagements.“ • Made suggestions regarding the development • “Overall experience is really cool-can provide good of adaptive thinking and improvements of the repetitions and learning.” LVC environment Participant Responses: Things to Work On By increasing the understanding of tactical • “Having constructive aircraft outside airspace does not help relationships, aircrew can modify their real-time tactical training with heavy civilian traffic because we do not thought processes using the results of new or better fight aircraft outside the airspace. This needs some heavy experiences provided by the LVC readiness admin discussion.” construct and technological capability. • “More in-aircraft troubleshooting capabilities or instruction.” Participant responses are a representative subset of a fuller set of comments Pillar Six Slides PA Cleared / Case Number 88ABW-2018-5740