Deep Representation and Reinforcement Learning Soumalya Sarkar, PhD - PowerPoint PPT Presentation

Deep Representation and Reinforcement Learning Soumalya Sarkar, PhD for Anomaly Detection and Control Senior Research scientist, UTRC in Multi-modal Aerospace Applications May 9 @ GTC 2017 This document contains no technical data subject to the EAR or the ITAR.

Our business units “UTRC is where you bring your toughest problems.” IIoT 2 This document contains no technical data subject to the EAR or the ITAR.

A global leader SALES BY GEOGRAPHY SALES BY TYPE $56B END MARKETS Military Aerospace Aftermarket Other & Space 12% 15% 2015 UTC Sales Asia 38% United 44% 20% States $3.9B 52% Pacific 56% 36% Original 27% Equipment Commercial Commercial Europe invested in R&D Manufacturing & Industrial Aerospace 3 This document contains no technical data subject to the EAR or the ITAR.

A global Cork, Ireland Established in 2010, presence focuses on energy, security and aerospace systems Shanghai, China Established in 1997, focuses on integrated buildings, fluid and mechanical systems Berkeley, CA Established in 2009, focuses on cyber physical systems Rome, Italy and embedded intelligence East Hartford, CT Joined UTC in 2012, focuses on model-based Founded in 1929, focuses on a design and embedded broad range of system engineering, systems engineering thermal, fluid, material, and informational sciences 4 This document contains no technical data subject to the EAR or the ITAR.

Focused on performance Thermal & Fluid Physical Sciences Systems Sciences – Advanced Materials – Advanced Laboratory for Embedded Systems – Acoustics – Applied Physics – Control Systems – Aerodynamics – Environmental Science – Cyber-Physical Systems – Aero-Thermal Testing – Materials Chemistry – Decision Support & Machine Intelligence – Combustion Science – Measurement Science – Electromagnetics & Networks – Propulsion Technology – Solid Mechanics – Power Electronics – Thermo-Fluid Dynamics – Surface Mechanics – Software Systems – Thermal Management – System Dynamics & Optimization 5 This document contains no technical data subject to the EAR or the ITAR.

Topics  Deep Representation Learning - DAE  Big PHM (Prognostics & Health Monitoring)  SHM (Structural Health Monitoring)  Deep Reinforcement Learning (DRL) for additive manufacturing This document contains no technical data subject to the EAR or the ITAR.

Deep Auto-Encoder (DAE) Multi-layer neural network based learner of non-linear representation of the data Input: Hidden representation: Sigmoid connecting two layers: W Parameters: Sigmoid function at reverse mapping of reconstruction layers: , 𝑋 ′ = 𝑋 𝑈 Where Cost function for back- propagation: • Parameter learning by Stochastic gradient descent (Hinton&Salakhutdinov, 2006; Bengio et al., 2007) • Variants: De-noising (RBM), variational etc • Static DAE instead of LSTM AE due to ease / speed of training This document contains no technical data subject to the EAR or the ITAR.

Topics  Deep Representation Learning - DAE  Big PHM (Prognostics & Health Monitoring)  SHM (Structural Health Monitoring)  Deep Reinforcement Learning (DRL) for additive manufacturing This document contains no technical data subject to the EAR or the ITAR.

Motivation Big, multi-modal & heterogeneous data; unsupervised visualization Data: ~100 sensors, ~200 dimensional condition data Size ~ TB Zero/ few labels Problem: Understanding / separating different missions or faults Challenges: Low-dimensional visualization, robust separation of faults (FDI) / mission, real-time application and generalizability This document contains no technical data subject to the EAR or the ITAR.

FDI approaches and challenges Previous Work and Challenges Methods of fault detection and identification Data-driven Model based Hybrid 1. Time, frequency, symbolic domain 1. Residual methods 1. Parity Equation Approach and features 2. Parity based wavelet based signal features 2. SVM, k-NN, artificial neural net based 3. Kalman filter based 2. PCA based system models learning systems • Lack of high-fidelity non-linear models, • Tedious hand-crafting (domain knowledge) of fault features, • Lack of scalability to large data, Deep Learning • Insufficient robustness to noise and • The presence of various operating modes, • Presence of multi-modal sensors for fault disambiguation This document contains no technical data subject to the EAR or the ITAR.

Database* for Validation Fault Detection and Identification Apparatus: A set of electromechanical actuators (EMA), constructed by Moog Corporation, were used by Balaban et. al. (Balaban et al., 2009, 2015). To increase the horizon of available operating conditions, flyable electromechanical actuator (FLEA) testbed was also constructed. 13 multi-modal sensors @100Hz : Actuator Z Position, Measured Load, Motor Current X-Y-Z, Motor Voltage X-Y, Motor Temperature X-Y-Z, Nut X-Y Temperature, Ambient Temperature. Baseline an 2 fault classes: 1. A jam fau lt injected via a mechanism mounted on the return channel of the ball screw that can stop circulation of the bearing balls through the circuit. 2. A spall fault injected by introducing cuts of various geometries via a precise electrostatic discharge process. The initial size and subsequent growth of these cuts were confirmed by using an optical inspection and measurement system. *open database available at NASA Dashlink, collected by Balaban et. al. (Balaban et al., 2009, 2015) This document contains no technical data subject to the EAR or the ITAR.

DAE architecture 650 (50x13) -> 256 -> 196 -> 136 -> 76 -> 14 -> 76 -> 136 -> 196 -> 256 -> 650 11-layer DAE Window size = 0.5 seconds, shifted by each time point This document contains no technical data subject to the EAR or the ITAR.

DAE Reconstruction Error Multi-modal Reconstruction Error Fault Detection and Identification Actual signals and reconstructed signals for Motor X voltage, Motor Y temperature, and load sensors (from top to bottom) with bottleneck layer of 14 dimension This document contains no technical data subject to the EAR or the ITAR.

Training and Parameter Learning Tuning bottleneck Layer Variation of normalized RMS error at the reconstructed output layer with increasing dimension of the bottleneck Individual sensor-wise reconstruction errors at the output layer for 3 different bottleneck layer dimensions This document contains no technical data subject to the EAR or the ITAR.

Fault Diagnostics ROC and Precision-Recall curves ROC curves via varying detection threshold on Precision-Recall curves for the same conditions as testing data for different bottleneck dimensions of data is unbalanced 11-layer DAE and few single layer AE models This document contains no technical data subject to the EAR or the ITAR.

Unsupervised Fault Disambiguation Disambiguation by Multi-dimensional Reconstruction Error Spider charts showing the NRMS error across different sensors during testing phase for nominal and fault scenarios This document contains no technical data subject to the EAR or the ITAR.

Why Deep Architecture? DAE Reconstruction Error increases fault separability with low over-fitting Spider charts of the average (over nominal and fault scenarios) NRMS error across different sensors during testing phase for (a) single hidden-layer AE with 512-dimensional bottleneck (b) proposed 11-layer DAE with 14-dimensional bottleneck This document contains no technical data subject to the EAR or the ITAR.

Why Deep Architecture? Multi-dimensional NRMS from DAE increases inter-fault distance at low dimension Clusters of two largest principal components obtained from PCA on 13-dimensional NRMS error distributions proposed 11-layer DAE with 14-dimensional bottleneck layer single hidden-layer AE model with 512-dimensional bottleneck layer This document contains no technical data subject to the EAR or the ITAR.

Discussions Fault separation even at a low dimension in an unsupervised way • trained directly on raw time series from heterogeneous sensors without feature hand-crafting and extensive data preprocessing. • A high fault detection rate ( ~97.8%) along with zero false alarm on a large set of realistic data (available on NASA DASHlink) • Disambiguation among different types of faults with high confidence in an unsupervised way • Proposed DAE more robust than single-hidden layer AE This document contains no technical data subject to the EAR or the ITAR.

Topics  Deep Representation Learning - DAE  Big PHM (Prognostics & Health Monitoring)  SHM (Structural Health Monitoring)  Deep Reinforcement Learning (DRL) for additive manufacturing This document contains no technical data subject to the EAR or the ITAR.