Modeling & Simulation, Testing & Validation GETTING AHEAD OF THE GAME: QUANTIFYING THE VALUE OF LABORATORY TESTING PRIOR TO FULL-UP SYSTEM LEVEL Joel Luna Mark Brudnak, PhD Sharon Snider Bryan LaRose Frontier Technology, Inc. Melissa Morgan Beavercreek, OH Dan Kosinski Mike Scott US Army TARDEC Warren, MI 7/31/2018 DISTRIBUTION STATEMENT A. Approved for public release; distribution unlimited. (OPSEC1189)
Modeling & Simulation, Background Testing & Validation • Automotive Reliability T&E Working Group, c. 2014 • Goal: Improve effectiveness and efficiency of government automotive reliability T&E through development of guidance, policy, training. • Initial Findings: – Insight from industry and lessons learned can be leveraged to deliver a more reliable and sustainable product to the soldier – Government full-up system level testing is not excessive, rather inefficient for reliability growth – Government reliability testing should be integrated with the systems engineering process to include developmental testing of components and subsystems with defined entrance criteria prior to FUSL testing (crawl – walk – run) – Tailor testing to align with full range of operational usage expectations rather than average usage (OMS/MP). 7/31/2018
Modeling & Simulation, Need for Development Testing Testing & Validation Reliability Test & Evaluation is laid over the system engineering & development process Development Evaluation B C (capability integration) (full up system testing) Currently full-up system level T&E is utilized to address continued system reliability shortfalls Planning assumes relatively mature system on entry. Practice shows many systems do not meet entry goals. Less testing for FUSL evaluation – More testing for development! 7/31/2018
Modeling & Simulation, Need Optimal Balance of Lab and Field Testing Testing & Validation Lab Field Testing Testing Pros: Cons: Pros: Cons: Early FM discovery Partial FM coverage Natural environment Late FM discovery Time compression No environment Established method Cost Subsystem system Repeatability/Control Full system test Less repeatability/ Lower cost reliability translation control Suitable mix Time Cost Risk Minimize: 7/31/2018
Modeling & Simulation, Lifecycle Test Optimization SBIR A16-085 Effort Testing & Validation 7/31/2018
Modeling & Simulation, Army Lifecycle Test Optimization (ALTO) Tool Testing & Validation Dashboard Input Test Monitoring Calculator Tools 7/31/2018
Modeling & Simulation, Sample ALTO Workflow Concept Testing & Validation 7/31/2018
Modeling & Simulation, Steps for Creating a Study in ALTO Testing & Validation 1. Create study file specific to the Unit Under Test (UUT) 2. Define initial planning factors and constraints (reliability, schedule, budget, risk) 3. Import/define data (e.g., FMECA data, subsystem/failure mode assignment, MS and FEF values) 4. (Optional) Explore characteristics of data 5. Obtain optimal test allocations between subsystems/system and phases using ALTO Optimizer 6. Review dashboard, determine areas of concern 7. Refine constraints and planning factors as needed, including revising schedule 8. Update data based on current test progress 9. Repeat 5-8 as needed 7/31/2018
Modeling & Simulation, Importing, Viewing, and Editing Data Testing & Validation Failure Mode, Component, or Subsystem Level Part Failure Rates Failure Rates Add New Failure Rates 7/31/2018
Modeling & Simulation, ALTO Dashboard for Quick Review and Re-Planning Testing & Validation Status for Subsystem, System DT and OT System Reliability Estimate Schedule Summary Make adjustments to Inspect and Resolve Issues 7/31/2018
Modeling & Simulation, Views to Quickly Make Adjustments to Test Plan Testing & Validation View Calculated Results Adjust Optimization Parameters Subsystem System Adjust Test Planning Inputs 7/31/2018
Modeling & Simulation, Test Monitoring Testing & Validation • The user can create any number of tests and units under test that contribute ultimately to the system reliability Test Miles by Test, Unit Under Test Test Details for each Test, Unit Under Test 7/31/2018
Modeling & Simulation, View Estimated and Test – System Reliability Testing & Validation Compare latest Test estimate with Planned estimate 7/31/2018
Modeling & Simulation, Next Steps Testing & Validation • Incorporate uncertainty and risk reduction in development testing (i.e., non-growth reliability testing) – Incorporate in multi-objective optimization • Develop consistent framework for merging test and design reliability estimate data – Designating test incidents that are considered failures – Computing and comparing uncertainty • Incorporate simulation to characterize uncertainty of final development MMBF • Conduct sample case demonstrating tool 7/31/2018
Modeling & Simulation, Testing & Validation Questions? Thank You! 7/31/2018
Modeling & Simulation, Army Lifecycle Test Optimization (ALTO) Tool Testing & Validation Features • Provide an innovative capability to conduct lifecycle test optimization – Defendable and repeatable results – Balance cost, schedule, and risk for test planners – Provide recommended solutions for optimizing test given time and money constraints – React to changes to time, cost, and risk in any phase of acquisition • Concept software functions for lifecycle test optimization – Import US Army data sources (FMECA data, Test Incident Data) – Edit/Enter reliability data, factors and constraints – Compute measures from input data using mathematical models – Provide immediate output for quick look analysis using calculators – Perform optimization with algorithms built into the software – Integrate with reliability calculators – Display metrics (tables, charts, etc.) – Produce an optimal balance of system and subsystem DT level testing • Implement a design that is flexible and adaptable – Implements multiple modeling approaches and can an extend to multiple data sources – Exports/imports data to interface with other applications 7/31/2018
Recommend
More recommend
