AnomalyDAE: Dual Autoencoder for Anomaly Detection on Attributed - PowerPoint PPT Presentation

-1- AnomalyDAE: Dual Autoencoder for Anomaly Detection on Attributed Networks Haoyi Fan 1 , Fengbin Zhang 1 , Zuoyong Li 2 Harbin University of Science and Technology 1 Minjiang University 2 isfanhy@hrbust.edu.cn

-2- Background https://webfoundation.org/2019/09/raising-the-bar-for-internet-access- introducing-meaningful-connectivity/ World Wide Web Social Network https://cointelegraph.com/news/south-africas-standard-bank-to-launch- https://www.sciencedirect.com/science/article/pii/S0969212610003035 permissioned-blockchain-for-overseas-exchange-services Biology Network Finance Transaction Network

-3- Background Anomaly Detection on the Attributed Network Anomaly Anomaly Latent Space

-4- Background Different types of anomalies Structure-inconsistent Structure-consistent Structure-inconsistent Attribute-consistent Attribute-inconsistent Attribute-inconsistent • Challenges: The cross-modality interactions between • the network structure and node attribute Neighbor-attention aware anomaly • Different neighbors contribute measuring differently for anomaly detection

-5- Background Numerous attributed network based anomaly detection methods have been proposed… Node Attribute LOF Breunig et al. 2000 SCAN Xu et al. 2007 FocusCO FocusCO Perozzi et al. 2014 LOF AMEN Perozzi et al. 2016 AMEN Radar Li et al. 2017 Radar ANOMALOUS Peng et al. 2018 ANOMALOUS Dominant Ding et al. 2019 Dominant … SCAN Network Structure Deep representation learning • framework on graph? The cross-modality interactions • between the network structure and node attribute?

-6- Problem Statement Notations 𝓗 : Attributed network 𝓦 : Set of nodes in network. 𝓕 : Set of edges in network. Problem 𝑁 : Number of nodes. 𝑂 : Dimension of attribute. Given 𝓗 = 𝓦, 𝓕, 𝐘 , learn a score function 𝐁 ∈ ℝ 𝑁×𝑁 : Adjacency matrix 𝑔: 𝓦 𝑗 ↦ 𝑧 𝑗 ∈ ℝ , to classify sample 𝑦 𝑗 based of a network. on the threshold 𝜇 : 𝐘 ∈ ℝ 𝑁×𝑂 : Attribute matrix of 𝑧 𝑗 = {1, 𝑗𝑔 𝑔(𝓦 𝑗 ) ≥ 𝜇, all nodes. 0, 𝑝𝑢ℎ𝑓𝑠𝑥𝑗𝑡𝑓. where 𝑧 𝑗 denotes the label of sample 𝑦 𝑗 , with 0 being the normal class and 1 the anomalous class.

-7- Method AnomalyDAE Structure Autoencoder Attribute Autoencoder

-8- Method AnomalyDAE Structure-level and attribute-level anomaly score

-9- Method AnomalyDAE Node Attention Attribute Generation

-10- Method Neighbor-attention Mechanism in Structure Autoencoder Initial feature transformation: ~𝓦 = 𝜏(𝐘𝐗 𝓦 (𝟐) + b 𝓦 (𝟐) ) 𝐚 𝐘, 𝐁 Importance weights: ~ ~ 𝓦 , 𝐚 𝓦 𝑓 𝑗,𝑘 = 𝑏𝑢𝑢𝑜 𝐚 𝑗 𝑘 ~ ~ = 𝜏( a T ⋅ [𝐗 𝓦 (𝟑) 𝐚 𝓦 ||𝐗 𝓦 (𝟑) 𝐚 𝓦 ]) ~𝓦 𝑗 𝑘 𝐚 Normalization: exp(𝑓 𝑗,𝑘 ൯ 𝛿 𝑗,𝑘 = σ 𝑙∈𝒪 𝑗 exp (𝑓 𝑗,𝑙 ቁ Neighbor-attention aware feature aggregation: ~ 𝐚 𝓦 𝓦 = σ 𝑙∈𝒪 𝑗 𝛿 𝑗,𝑙 ⋅ 𝐚 𝓦 𝐚 𝑗 𝑙

-11- Method Cross-modality Interactions Capturing in Attribute Autoencoder 𝐚 𝓦 Attribute reconstruction: ^ = 𝐚 𝓦 (𝐚 𝓑 ) T 𝐘 𝐚 𝓑

-12- Method Loss and Anomaly Score Loss Function: ^ ^ 2 + (1 − 𝛽)||(𝐘 − 𝐘 2 ℒ 𝑠𝑓𝑑 = 𝛽||(𝐁 − 𝐁 ) ⊙ 𝜾|| 𝐺 ) ⊙ 𝜽|| 𝐺 𝑝𝑢ℎ𝑓𝑠𝑥𝑗𝑡𝑓., 𝜽 𝑗,𝑘 = { 1 𝑗𝑔 𝐘 𝑗,𝑘 = 0, 𝜾 𝑗,𝑘 = {1 𝑗𝑔 𝐁 𝑗,𝑘 = 0, 𝜃 𝑝𝑢ℎ𝑓𝑠𝑥𝑗𝑡𝑓. 𝜄 Anomaly Score: ^ ^ 2 + (1 − 𝛽)||(𝐘 − 𝐘 2 𝑇𝑑𝑝𝑠𝑓 = 𝛽||(𝐁 − 𝐁 ) ⊙ 𝜾|| 𝐺 ) ⊙ 𝜽|| 𝐺 Structure-level Attribute-level Anomaly Measure Anomaly Measure

-13- Method Loss and Anomaly Score Loss Function: ^ ^ 2 + (1 − 𝛽)||(𝐘 − 𝐘 2 ℒ 𝑠𝑓𝑑 = 𝛽||(𝐁 − 𝐁 ) ⊙ 𝜾|| 𝐺 ) ⊙ 𝜽|| 𝐺 𝑝𝑢ℎ𝑓𝑠𝑥𝑗𝑡𝑓., 𝜽 𝑗,𝑘 = { 1 𝑗𝑔 𝐘 𝑗,𝑘 = 0, 𝜾 𝑗,𝑘 = {1 𝑗𝑔 𝐁 𝑗,𝑘 = 0, 𝜃 𝑝𝑢ℎ𝑓𝑠𝑥𝑗𝑡𝑓. 𝜄 Anomaly Score: ^ ^ 2 + (1 − 𝛽)||(𝐘 − 𝐘 2 𝑇𝑑𝑝𝑠𝑓 = 𝛽||(𝐁 − 𝐁 ) ⊙ 𝜾|| 𝐺 ) ⊙ 𝜽|| 𝐺 Solution for Problem: 𝑧 𝑗 = {1, 𝑗𝑔 𝑔 𝓦 𝑗 ≥ 𝜇, 0, 𝑝𝑢ℎ𝑓𝑠𝑥𝑗𝑡𝑓. 𝜇 =Distribution( 𝑇𝑑𝑝𝑠𝑓 )

-14- Experiment Datasets Baselines Evaluation Metric LOF Breunig et al. 2000 AUC ( A rea U nder a SCAN Xu et al. 2007 receiver operating AMEN Perozzi et al. 2016 characteristic C urve) Radar Li et al. 2017 ANOMALOUS Peng et al. 2018 Dominant Ding et al. 2019 …

-15- Experiment Results 19.68% 22.32% 15.11% At least 15.11% ~ 22.32% AUC improvement!

-16- Experiment Results Robust and Effective!

-17- Conclusion Traditional machine learning based methods • perform poor for feature learning on large graph. Traditional deep graph model cannot effectively • capture the cross-modality interactions between the network structure and node attribute. We propose a deep joint representation learning • framework via a dual autoencoder to capture the complex cross-modality interactions between the network structure and node attribute.

-18- Reference [LOF] Breunig, Markus M., et al. "LOF: identifying density-based local • outliers." KDD. 2000 . [SCAN] Xu, Xiaowei, et al. "Scan: a structural clustering algorithm for • networks." KDD. 2007 . [FocusCO] Perozzi, Bryan, et al. "Focused clustering and outlier detection in • large attributed graphs." KDD. 2014 . [AMEN] Perozzi, Bryan, and Leman Akoglu. "Scalable anomaly ranking of • attributed neighborhoods." SIAM, 2016 . [Radar] Li, Jundong, et al. "Radar: Residual Analysis for Anomaly Detection in • Attributed Networks." IJCAI. 2017 . [GAT] Veličković , Petar, et al. "Graph attention networks." ICLR. 2018 . • [ANOMALOUS] Peng, Zhen, et al. "ANOMALOUS: A Joint Modeling Approach • for Anomaly Detection on Attributed Networks." IJCAI. 2018 . [Dominant] Ding, Kaize, et al. "Deep anomaly detection on attributed • networks." SIAM, 2019 .

-19- Thanks Thanks for listening! Contact: isfanhy@hrbust.edu.cn Home Page: https://haoyfan.github.io/