Tensor Decompositions for ensor Decompositions for Big Multi-aspect - PowerPoint PPT Presentation

Tensor Decompositions for ensor Decompositions for Big Multi-aspect Data Big Multi-aspect Data Analytics Analytics Evangelos Evangelos (V (Vagelis) agelis) Papalexakis Papalexakis UC Riverside Second W Second Workshop of Mission-Critical Big Data Analytics (MCBDA 2017) orkshop of Mission-Critical Big Data Analytics (MCBDA 2017)

Roadmap Roadmap 2 E. Papalexakis @ MCBDA 17

Multi-Aspect Data?? 3 E. Papalexakis @ MCBDA 17

Multi-V Multi-View Social iew Social Networks Networks 4 E. Papalexakis @ MCBDA 17

Social Network Matrix Social Network Matrix 5 E. Papalexakis @ MCBDA 17

Outer Pr Outer Product oduct 1x3 b T 3 3 4 4 5 5 By definition: 3 3 4 4 5 5 1 1 = = Rank-one matrix Rank-one matrix 6 8 8 10 6 10 2 2 10 a a 2x1 2x3 Matrix Rank Matrix Rank = Min # of rank-one matrices that add up to that matrix 6 E. Papalexakis @ MCBDA 17

Matrix Decomposition Matrix Decomposition Decomposition into rank-1 = matrices/components 7 E. Papalexakis @ MCBDA 17

Singular Value Decomposition Singular V alue Decomposition v 1 v k T T T σ 1 σ k 1 k = X X + … + u 1 u k 1 • If k = rank(X) then we have equality • If k < rank(X) we have the best rank k approximation that minimizes the squared error (Eckart Young Theorem) 8 E. Papalexakis @ MCBDA 17

What if we have mor What if we have more than 1 view e than 1 view of the network? of the network? If we aggregate, we ignore important structure!! 9 E. Papalexakis @ MCBDA 17

Tensors ensors • Multi-dimensional matrices • Long list of applications: Chemometrics, Psychometrics, Signal Processing, Data Mining X X 10 E. Papalexakis @ MCBDA 17

What ar What are we looking for? e we looking for? Blocks Blocks within the data Subsets / co-clusters of: 1) Users (“senders”) 2) Users (“receivers”) X X 3) Means of communication 11 E. Papalexakis @ MCBDA 17

Thr Three-way Outer Pr ee-way Outer Product oduct c 2x3 2x1 3 3 1x3 9 12 12 15 9 15 2 2 b 18 21 21 30 18 30 3 4 4 5 3 5 6 8 6 8 10 10 1 1 = = 2x3x2 rank-1 12 16 12 16 20 20 2 2 10 a a tensor 2x1 2x3 Blocks ar Blocks are rank-1 tensors e rank-1 tensors 12 E. Papalexakis @ MCBDA 17

CP/P CP/PARAF ARAFAC Decomposition AC Decomposition c 1 c K b 1 b K ≈ X X + … + a 1 a K a k � b k � c k k 2 X A , B , C k X � min F k outer product 13 E. Papalexakis @ MCBDA 17

G RAPH RAPH F USE USE PARAFAC with c 1 c K Sparse Latent Factors Step 1 Step 1 [Papalexakis et al. [Papalexakis et al. b 1 b K ≈ X + … IEEE ICASSP 2011, IEEE IEEE ICASSP 2011, IEEE + TSP 2013 ] TSP 2013 ] a 1 a K Assign users to communities Step 2 Step 2 (max component membership) For users with no assignment Step 3 Step 3 create K+1 th (umbrella) community Output: 1. Assignment of users to communities 2. Influence of a view to each community [Papalexakis et al. Fusion 2013 Fusion 2013] 14 E. Papalexakis @ MCBDA 17

DBLP Multi-V DBLP Multi-View Graph iew Graph (a) citation (b) co-auth. (c) co-term • Assignment of authors to research communities • Measure NMI (Normalized Mutual Information) • Baselines ² Spectral clustering on sum of matrices / views ² Linked Matrix Factorization [Tang et al. ICDM 2009] • G RAPH F USE outperforms “2D” baselines [Papalexakis et al. Fusion 2013 Fusion 2013] 15 E. Papalexakis @ MCBDA 17

Time-Evolving Graphs as T ime-Evolving Graphs as Tensors ensors …! …! …! t K ! t 1 ! t 2 ! t 3 ! Detect anomalies / real-life events 16 E. Papalexakis @ MCBDA 17

Time-Evolving Graphs as T ime-Evolving Graphs as Tensors ensors • Tensor Decomposition will give us ² Communities in the Graph & ² Their evolution over time Change Detection 1 ICDE, SIGMOD, VLDB Point of CIKM,ECIR,ICDE,ICDM,IJCAI,JCDL,KDD,SIGIR, WWW 0.8 change 0.6 0.4 0.2 0 1986 1988 1990 1992 1994 1996 1998 2000 2002 2004 2006 Year Fig. 3. In this Figure we demonstrate how T ENSOR S PLAT is able to perform change detection. In particular, we observe two components in which a well- known professor appears as an author; the first component mainly contains Databases conferences, whereas the second contains Data Mining conferences. The dashed red line indicates the point of change in research direction. Koutra, Papalexakis, Faloutsos, “TENSORSPLAT: Spotting Latent Anomalies in Time” 17 E. Papalexakis @ MCBDA 17

Neur Neurosemantics osemantics … � � � � �� � � � � � � � � � � � � �� �� �� “Does it fly?” (y/n) “Does it fly?” (y/n) “Does it bite?” (y/n) “Does it bite?” (y/n) fMRI MEG Real and predicted MEG brain activity Real and predicted MEG brain activity 0.4 0.3 0.3 ed: 0.25 0.25 0.3 0.2 0.2 0.2 0.15 0.15 0.1 0.1 0.1 0.05 0 0.05 0 20 40 0 20 40 0 20 40 18 E. Papalexakis @ MCBDA 17 � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �

Neur Neurosemantics osemantics • Semantically coherent brain regions? • Similarities/differences between subjects? • How is language processed in the brain? … � � � � �� � � � � � � � � � � � � �� �� �� “Does it fly?” (y/n) “Does it fly?” (y/n) “Does it bite?” (y/n) “Does it bite?” (y/n) Real and predicted MEG brain activity Real and predicted MEG brain activity 0.4 0.3 0.3 ed: 0.25 0.25 0.3 0.2 0.2 0.2 0.15 0.15 0.1 0.1 0.1 0.05 0 0.05 0 20 40 0 20 40 0 20 40 19 E. Papalexakis @ MCBDA 17 � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �

Airplane Airplane Puppy Puppy Dog Dog Combining measurements for Combining measur ed: ed: multiple subjects multiple subjects E. Papalexakis @ MCBDA 17 ed: ed: ements for ed: ed: ed: ed: 20

Modeling Brain Data as Tensor Modeling Brain Data as T Airplane Airplane Puppy Puppy Dog Dog E. Papalexakis @ MCBDA 17 fMRI voxels fMRI voxels ed: ed: ed: ed: ed: ed: ed: ed: ensor 21

Airplane Airplane Puppy Puppy Dog Dog CP/P CP/PARAF ed: ed: a r , b r , c r k X � ed: ed: ARAFAC Decomposition min ed: ed: X X ed: ed: AC Decomposition E. Papalexakis @ MCBDA 17 = = r =1 X R a 1 1 1 1 1 a r � b r � c r k 2 c 1 b 1 ed: F + + a 2 1 1 c 2 b 2 22

� � � � �� Semantic Information Semantic Infor mation ✔ Is it alive? ✔ Does it bite? Does it fly? ✔ Can you buy it? Is it smaller than a golf ball? … • Human readable description of the noun • Useful information to guide the analysis • Can have different semantic features (corpus statistics, knowledge base features) 23 E. Papalexakis @ MCBDA 17 � � � � � � � � � � � �

Y Tensor W ✔ ensor With Side Infor ✔ ✔ [Papalexakis Papalexakis et al. SDM 2014 ✔ ✔ ith Side Information E. Papalexakis @ MCBDA 17 ✔ ✔ ✔ et al. SDM 2014] Airplane Airplane Puppy Puppy Dog Dog ed: ed: mation ed: ed: ed: ed: X X ed: ed: 24

Y a 1 1 1 1 1 ✔ Proposed Modeling: Coupled Pr ✔ ✔ = = ✔ d 1 Matrix-T Matrix-Tensor Factorization oposed Modeling: Coupled ✔ ✔ ✔ + + ✔ a 2 ✔ ✔ ✔ 1 1 Airplane Airplane Puppy Puppy ✔ Dog Dog d 2 ensor Factorization E. Papalexakis @ MCBDA 17 ✔ ed: ed: ed: ed: ed: ed: X X ed: ed: = = 1 1 1 1 a 1 c 1 ed: b 1 + + a 2 1 1 c 2 b 2 25

Y a 1 1 1 1 1 ✔ Pr Proposed Modeling: Coupled ✔ ✔ = = ✔ d 1 Matrix-Tensor Factorization Matrix-T oposed Modeling: Coupled ✔ ✔ ✔ + + ✔ a 2 ✔ ✔ ✔ a r , b r , c r , d r k X � 1 1 Airplane Airplane min Puppy Puppy ✔ Dog Dog d 2 ensor Factorization E. Papalexakis @ MCBDA 17 ✔ ed: ed: ed: ed: Tensor part r =1 X ed: ed: R X X ed: ed: a r � b r � c r k 2 = = 1 1 1 1 a 1 c 1 F + k Y � ed: b 1 + + Matrix part r =1 X a 2 1 1 R c 2 a r d T b 2 r k 2 26 F

Pr Pre-motor Cortex e-motor Cortex Nouns Q beetle c glass tomato bell ✔ Unsupervised Nou Questions Nou ✔ Agrees with Neuroscience can you pick it up? can you hold it in one hand? is it smaller than a golfball?’ 6 0.05 0.05 50 50 0.04 100 0.04 100 0.03 150 0.03 150 200 0.02 200 0.02 250 Pre$motor(cortex( 0.01 250 0.01 Premotor Cortex 300 0 300 50 100 150 200 250 0 50 100 150 200 250 27 E. Papalexakis @ MCBDA 17

Multi-Aspect Data Everywher Multi-Aspect Data Everywhere! e! Social Networks Social Networks & Urban Comp. & Urban Comp. � � � � �� � � � � � � � � �� �� � � � � �� Neur Neurosemantics osemantics … W Web eb is president Knowledge Knowledge … � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 28 � � E. Papalexakis @ MCBDA 17 � � � � � �

Roadmap Roadmap 29 E. Papalexakis @ MCBDA 17