Memristor Based Autoencoder for Unsupervised Real-Time Network - PowerPoint PPT Presentation

Memristor Based Autoencoder for Unsupervised Real-Time Network Intrusion and Anomaly Detection Md. Shahanur Alam, B. Rasitha Fernando, Yassine Jaoudi, Chris Yakopcic, Raqibul Hasan, Tarek M. Taha, and Guru Subramanyam Dept. Of Electrical and Computer Engineering , University of Dayton, Dayton, OH, USA

Outline • Introduction • Anomaly Detection Methods and Applications • Motivation and Challenges • Proposed Anomaly Detection System • Results of Intrusion and Anomaly Detection System • Summary • Future work M. S. Alam et. al. 2

Introduction • Network Intrusion • Intrusion Detection system • SNORT • What if new unknown packet comes? E.g. ‘Zero Day’ Neural Network SNORT Positive Normal Router Positive + Zero Day Anomaly Negative Block diagram of the neural network-based intrusion detection system M. S. Alam et. al. 3

Introduction (Contd.) Neural Network Vs Power Consumption ≈ 200W IoTs and Edge Devices 𝑁(𝑟) = 𝑒𝜚 𝑒𝑟 • Memristive system could be a solution Memristor M. S. Alam et. al. 4

Anomaly Detection Methods and Applications What are the anomalies? Anomaly detection Methods: • Abnormalities/outliers • Unsupervised (AE, GAN, RNN, LSTM etc) • Supervised (DNN, CNN) 𝑍 • Hybrid model (AE+SVM) 𝐸 1 𝑂 2 𝐸 2 • One-Class Neural Network Applications: 𝑂 1 • Cyber-Intrusion Detection • Malware Detection 𝐸 3 • Internet of Things (IoTs) Big Data Anomaly Detection • Fraud Detection 𝑌 • Medical Anomaly Detection Illustration of anomalies in two-dimensional data set • Industrial Damage Detection M. S. Alam et. al. 5

Motivation and Challenges Motivation: • Neural Network implementation for IoTs and edge devices • Detection of anomalies in real-time Challenges: • Boundary between normal and malicious is not explicitly defined • Continual learning and the catastrophic forgetting M. S. Alam et. al. 6

Dataset Preprocessing Normal Packet 0,tcp,ftp_data,SF,491,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,2,0,0, 0,0,1,0,0,150,25,0.17,0.03,0.17,0,0,0,0.05,0,normal,20 Malicious Packet • NSL-KDD network dataset  KDD Cup’99 dataset 0,tcp,ftp_data,SF,334,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,2,2,0,0, • 0,0,1,0,0,2,20,1,0,1,0.20,0,0,0,0, warezclient,15 Training data has125,973 packets, 23 different data types Preprocessed Normal Packet • 43 attributes, consists numerical and alphanumeric data 0,0.5,0.188,0.629,3.55 𝑓 −7 ,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.003 • 91,0.00391,0,0,0,0,1,0,0,0.588,0.098,0.17,0.03,0.17,0,0,0,0.05 Preprocessed and sorted out the packets ,0,0,0.9523 • Network is pretrained with 90% of Normal Preprocessed Malicious Packet 0,0.5,0.188,0.629,2.42 𝑓 −7 ,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0.003 • Tested with 10% normal and 10% of total malicious data 91,0.0039,0,0,0,0,1,0,0,0.0078,0.078,1,0,1,0.2,0,0,0,0,1,0.714 M. S. Alam et. al. 7

Proposed Anomaly Detection System System Prototype Model Autoencoder (AE) Neural Network w 1(i,j) w' 2(k,j) h 1,1 h 3,1 w' 1(j,i) w 2(j,k) Enterprise Network x 1 h 1,2 h 3,2 x' 1 Router x 2 h 2,1 h 1,3 h 3,3 x' 2 AE-1: Pretrained Section . . . SNORT . . . . . . h 1,4 h 3,4 1 2 h 2,k 3 . . . . . . Normal Data x i 4 . . . x' i . . . . . . . . . . . . . . . . . . . . . . . . h 1,j h 3,j . . . . . . h 2,10 Positive . . . . . . Positive Malicious Data Negative . . . . . . x 41 x' 41 h 1,90 h 3,90 Bottle Neck . . . Positive=Normal + ‘zero day’ Encoder Decoder . . . . . . packets . . . . . . . . . Known . . . . . . . . . . . . Unknown 41 → 90 → 10 → 90 → 41 AE-2:Real-Time Training • AE learns to regenerate the input data at output • AE can reduce the dimension of input data Intrusion And Anomaly Detection System with AE neural Network 8 M. S. Alam et. al.

Memristive Neural Network and Crossbar Circuit DOT Product: x 1 𝑂+1 𝑦 𝑗 × 𝜏 𝑗𝑘 + − 𝜏 𝑗𝑘 − 𝑘 = σ 𝑗=1 𝐸𝑄 (1) x 2 Synapse x 3 Sigmoid Approximation: . . . 1 𝑔 𝑦 = (2) 1+𝑓 −𝑦 . . . . . .  1, 𝑦 > 2  + x N − = 0.25𝑦 + 0.5, 𝑦 ≤ 2 𝑕 𝑦 = ቐ (3) 0, 𝑦 < 2 x N+1 β Memristor R + - A A A M A (a) Single Neuron 1 2 3 R R f + - y M y 2 y 3 y 1 (c) y j Memristor Crossbar Circuits (b) Ideal and approximate Sigmoid Function 9 M. S. Alam et. al.

Training of the Network • apply 𝑦 𝑗 • crossbar computes the dot product 𝐸𝑄 𝑘 • output signal 𝑧 𝑘 𝑘 = 𝑦 𝑗 − 𝑧 𝑘 𝑔 ′ 𝐸𝑄 • error : 𝜀 𝑘 𝑘 = σ 𝑙 𝜀 𝑙 𝑥 𝑙,𝑘 𝑔 ′ 𝐸𝑄 • backpropagate the error 𝜀 𝑘 in each hidden layer • update the weights according 𝜀 𝑘 as Δ𝑥 𝑘 = 𝜃𝜀 𝑘 𝑦 σ(𝐸−𝐸 𝑛 ) 2 1 𝑘 ) 2 and 𝐸 𝑇𝐸 = • σ(𝑌 𝑗 − 𝑍 calculate 𝐸 𝑛 = 𝑂 𝑂 M. S. Alam et. al. 10

System Flowchart of Anomaly Detection System 𝒇 = 𝒇 𝟏 + Y AE-1 D= σ(𝒀′ 𝒋 − 𝒁 𝒋 ) 𝟑 Data ( 𝒀 ’) Forward σ(𝒀′ 𝒋 − 𝒁 𝒋 ) 𝟑 Enterprise Network Router AE-1: Pretrained Section Update Weight of AE-2 SNORT ∆= 𝑬 − 𝑬 𝒏 1 Unknown 2 3 Normal Data 4 ? . . . . . . . . . . . . . . . . . . . . . . . . Positive For . . . . . . Positive Malicious Data Negative ∆> 𝑬 𝑻𝑬 , 𝑴 = 𝟐 For & ∆ 𝟐 > 𝑬 𝑻𝑬𝟐 , 𝒗𝒐𝒍𝒐𝒑𝒙𝒐 & ∆< 𝑬 𝑻𝑬 , 𝑴 = 𝟏 . . . Positive=Normal + ‘zero day’ . . . ∆ 𝟐 < 𝑬 𝑻𝑬𝟐 , 𝒍𝒐𝒑𝒙𝒐 . . . packets . . . . . . . . . Known . . . . . . . . . . . . Unknown 𝒇′ = 𝒇 𝟏 + Y’ AE-2 AE-2:Real-Time Training 𝑴 = 𝟐 /0 σ(𝒀′ 𝒋 − 𝒁′ 𝒋 ) 𝟑 ∆ 𝟐 = 𝑬′ − 𝑬′ 𝒏 Forward ? Anomaly Detection System Flowchart of Real-time Anomaly detection System 11 M. S. Alam et. al.

Pretraining of Autoencoder-1 (AE-1) a. b. Training Error (MSE) in software and memristor X-bar Input feature and regenerated feature of a sample through (AE-1) 12 M. S. Alam et. al.

Intrusion Detection Accuracy a. b. Intrusion detection Accuracy (AE-1) False Detection (Malicious + Normal) Pretraining Epochs Global Accuracy 𝑶 𝑵𝑶 𝑶 𝑶𝑵 𝑶 𝑮 Case 𝐵𝑑𝑑𝑣𝑠𝑏𝑑𝑧 = 𝑂 𝑡 −𝑂 𝐺 × 100% 𝑂 𝑡 Software 50 95.22% 56 546 602 Memristor 50 92.91% 65 868 933 M. S. Alam et. al. 13

Intrusion Detection Accuracy (contd.) a. b. Malicious Packet Vs Epochs Malicious Packet Detection Accuracy Vs Epochs 14 M. S. Alam et. al.

Real-time learning and anomaly detection Real-Time Anomaly Detection: 1 , 𝑦 2 2 , 𝑦 2 3 , 𝑦 2 1 , 𝑦 1 2 , 𝑦 1 3 , … 𝑈 1 = 𝑦 1 1 , 𝑦 2 1 , 𝑦 3 1 , 𝑦 1 2 , 𝑦 2 2 , … 2 , 𝑦 3 𝑈 2 = 𝑦 1 1 , 𝑦 2 1 , 𝑦 3 1 , 𝑦 4 1 , 𝑦 1 2 , 𝑦 2 2 , 𝑦 3 2 , 𝑦 4 2 , … 𝑈 3 = 𝑦 1 Enterprise Network 1 , 𝑦 2 1 , 𝑦 3 1 , 𝑦 4 1 , 𝑦 5 1 , 𝑦 1 2 , 𝑦 2 2 , 𝑦 3 2 , 𝑦 4 2 , 𝑦 5 2 , … 𝑈 4 = 𝑦 1 Router 𝑦 1 = 𝑜𝑝𝑠𝑛𝑏𝑚 , 𝑦 2 = 𝑜𝑓𝑞𝑢𝑣𝑜𝑓, 𝑦 3 = 𝑡𝑏𝑢𝑏𝑜, 𝑦 4 = AE-1: Pretrained Section SNORT 𝑗𝑞𝑡𝑥𝑓𝑓𝑞, 𝑦 5 = 𝑐𝑏𝑑𝑙 1 2 3 Normal Data 4 . . . . . . . . . . . . . . . . . . . . . . . . Positive . . . . . . Positive Malicious Data Negative . . . Positive=Normal + ‘zero day’ . . . . . . packets . . . . . . . . . Known . . . . . . . . . . . . Unknown AE-2:Real-Time Training Anomaly Detection System Anomaly Detection in real-time M. S. Alam et. al. 2/23

Power, Area and Timing Analysis Parameter Training Data Recognition Data 𝑆 𝑝𝑔𝑔 = 1 × 10 7 Ω, 𝑆 𝑝𝑜 = 5 × 10 4 Ω • Area (mm 2 ) • 0.00271 0.00271 Wire Resistance =5 Ω , 𝑊 𝑛𝑓𝑛 = 1.3𝑤𝑝𝑚𝑢 • Transistor Feature Size : F= 45nm Power (mW) 20.6 7.56 Op-amp power = 3 × 10 −6 𝑥𝑏𝑢𝑢 • Time (µs)/sample 4.02 0.384 Transistor Size= 50𝐺 2 • Energy (nJ)/One Sample 82 2.90 Memristor area= 1 × 10 4 𝑜𝑛 2 • M. S. Alam et. al. 16

Summary • Introduced the problem and proposed a possible solution • Presented the Autoencoder with memristor X-bar and the functionalities • Overall accuracy 92.91% with malicious packet detection accuracy 98.89% • Presented real-time anomaly detection system • Explained the power and energy requirement M. S. Alam et. al. 17

Current and future work • Multiple autoencoders for intrusion and malware detection • Incremental learning algorithm & unseen class detection M. S. Alam et. al. 18

THANK YOU Questions ? M. S. Alam et. al. 19