Dyn ynamic mic Pr Processes esses ove ver In Informat matio - PowerPoint PPT Presentation

idea adoption/disease spread/viral marketing D S means 1:00pm S follows D David vid 1:10pm Sophie hie 1:18pm 1:15pm Christine ine Bob 1:25pm Jacob ob 33

Scenario I: idea adoption D S means 1:00pm D D is source S follows D 𝑂 𝐸 𝑢 = 1 𝑢 David vid 1:10pm Sophie hie Terminating process adopt product only once B C 𝑂 𝐶 𝑢 𝑂 𝐷 𝑢 Followee Not yet adopted adopted 1:18pm 1:15pm Christine ine Bob 𝐵 𝐾𝐷 𝐵 𝐾𝐶 ℎ 𝐾∗ 𝑢 = 𝐵 𝐾𝐶 𝑢 1 − 𝑂 𝐾 𝑢 𝑂 𝐶 𝑢 1:25pm + 𝐵 𝐾𝐷 𝑢 1 − 𝑂 𝐾 𝑢 𝑂 𝐷 𝑢 J Jacob ob 𝑂 𝐾 𝑢 34

Cascades from D in 30 mins 1:00pm 1:08pm David vid Sophie hie 1:17pm Bob 1:25pm Jacob ob 1:48pm Christine ine 35

Cascades from D in 30 mins 1:00pm 2:00pm 1:03pm David vid David vid 2:08pm Sophie hie 1:17pm Sophie hie Bob 1:25pm Jacob ob 2:42pm 2:37pm 1:48pm Christine ine Bob Christine ine 2:53pm Jacob ob 36

Cascades from D in 30 mins 1:00pm 2:00pm 7:00pm 1:03pm David vid David vid David vid 7:06pm 2:08pm Sophie hie Sophie hie 1:17pm Sophie hie Bob 7:17pm 1:25pm Christine ine Jacob ob 7:32pm 2:42pm 2:47pm Bob 1:48pm Christine ine Bob 7:50pm Christine ine 2:53pm Jacob ob 37 Jacob ob

Cascades from D in 30 mins 1:00pm 2:00pm 7:00pm 1:03pm David vid David vid David vid 7:06pm 2:08pm Sophie hie Sophie hie 1:17pm Sophie hie Bob 7:17pm 1:25pm Christine ine Jacob ob 7:32pm 2:42pm 2:47pm Bob 1:48pm Christine ine Bob 7:50pm Christine ine 2:53pm Jacob ob 38 Jacob ob

Cascade Data Cascade: a sequence of (node, time) pairs for a particular piece of news Cascades can start from different sources Cascade 2 Cascade 1 Cascade 3 User 1 𝑢 User 2 𝑢 User 3 𝑢 … User n: 𝑢 𝑢 1 , 𝑢 2 , 𝑢 3 , … , 𝑢 𝑜 (𝑢 1 , 𝑢 2 , 𝑢 3 , … , 𝑢 𝑜 ) 𝑢 1 , 𝑢 2 , 𝑢 3 , … , 𝑢 𝑜 39

Dyn ynamic mic Pr Processes esses ove ver In Informat matio ion n Netwo works rks Rep epre rese sentat ntation, ion, Modeli deling ng, , Le Learning ning and d Infer erence ence Modeling: Coevolution 40

Information diffusion and network coevolution 1pm, D: D S 1:45pm Cool paper means S follows D David vid 2pm, D: Nice idea 1:10pm, @D: Indeed Tina Sophie hie 1:18pm, @S @D: 1:15pm, @S @D: Very useful Classic Bob Christine ine Olivi via Jacob ob 2:03pm, @D: Agree 1:35pm @B @S @D: Indeed brilliant Farajtabar et al. NIPS 2015 41

Information diffusion and network coevolution 1pm, D: Cool paper (D, D, 1:00) 1:45pm (J, D, 1:45) David vid Link creation event sequence Sophie hie (J, J) 𝑢 5:25pm (J, S, 5:25) (J, D) 4pm, B: (B, B, 4:00) (J, S) It snows … Christine ine Bob Tweet/retweet event sequence (J, J) 𝑢 1:35pm @B @S @D: (J, D, 1:35) Indeed brilliant (J, D) Jacob ob 4:10pm, @B: (J, B, 4:10) Beautiful (J, B) 5pm, J: … (J, J, 5:00) Going out 42

Targeted retweet (D, D) D’s own initiative ℎ 𝐸∗ 𝑢 = 𝜃 David vid Sophie hie (C, D) (B, D) 𝑂 𝐷𝐸 𝑢 𝑂 𝐶𝐸 𝑢 Mutually-exciting process Christine ine Bob High if followees retweet frequently 𝐵 𝐾𝐷 𝑢 𝐵 𝐾𝐶 𝑢 ℎ 𝐾𝐸∗ 𝑢 = 𝛾 𝐸 𝐵 𝐾𝐶 𝑢 exp − 𝑢 ⋆ 𝑒𝑂 𝐶𝐸 𝑢 (J, D) + 𝛾 𝐸 𝐵 𝐾𝐷 𝑢 exp − 𝑢 ⋆ 𝑒𝑂 𝐷𝐸 𝑢 Jacob ob 𝑂 𝐾𝐸 𝑢 43

Information driven link creation (J, D) 1:45pm 𝐵 𝐾𝐸 (𝑢) David vid 𝐾 ’s random exploration 𝛿 𝐾𝐸∗ 𝑢 = 1 − 𝐵 𝐾𝐸 𝑢 ⋅ 𝜈 𝐾 + 𝛽 𝐸 exp − 𝑢 ⋆ 𝑒𝑂 𝐾𝐸 𝑢 Sophie hie Check whether Retweet 𝐸 the link already there Self-exciting process Christine ine Bob Terminating process no link and retweet often (J, D) Jacob ob 𝑂 𝐾𝐸 𝑢 44

Joint model of retweet + link creation Diffusion network Diffusion network 𝑩 𝑢 ∈ {0,1} 𝑩 𝑢 ∈ {0,1} Alter Support Link creation Link creation Information diffusion Information diffusion process 𝑶 𝑢 ∈ 0 ∪ 𝑎 + process 𝑶 𝑢 ∈ 0 ∪ 𝑎 + process process Terminating Mutually-exciting Drive process process 45

Simulation 46

Link creation parameter controls network type 𝛽 𝐸 = 0 𝛽 𝐸 large Erdos-Renyi random networks Scale-free networks 47

Shrinking network diameters Generate networks with small shrinking diameter Small connected components merge Diameter shrinks 48

Cascade patterns: structure Generate short and fat cascades as 𝛽 increases 𝛾 = 0.2 49

Dyn ynamic mic Pr Processes esses ove ver In Informat matio ion n Netwo works rks Rep epre rese sentat ntation, ion, Modeli deling ng, , Le Learning ning and d Infer erence ence Modeling: Collaborative Dynamics 50

Collaborative dynamics 𝑝 1 𝑝 2 Low rank 𝑝 3 𝑝 4 𝜈 𝐸𝑝 1 𝜈 𝐸𝑝 4 … David id ⋮ ⋱ ⋮ 𝜈 𝐾𝑝 1 𝜈 𝐾𝑝 4 … 𝛽 𝐸𝑝 1 𝛽 𝐸𝑝 4 Sophi phie … ⋮ ⋱ ⋮ 𝛽 𝐾𝑝 1 𝛽 𝐾𝑝 4 … Chris istine tine Self-exciting process Tend to go to the same store Jacob ob again and again ℎ 𝐸𝑝 1 ∗ 𝑢 = 𝜈 𝐸𝑝 1 + 𝛽 𝐸𝑝 1 𝐸𝑝 1 | exp −|𝑢 − 𝑢 𝑗 𝐸𝑝1 ∈𝓘 𝑢 𝐸𝑝1 𝑢 𝑗 51

Dyn ynamic mic Pr Processes esses ove ver In Informat matio ion n Netwo works rks Rep epre rese sentat ntation, ion, Modeli deling ng, , Le Learning ning and d Infer erence ence Learning: Sparse Networks 52

Hidden diffusion networks Estimate the Estimate the diffusion diffusion networks? networks? 53

Parametrization of idea adoption model D S D is source means 1:00pm D 𝑂 𝐸 𝑢 = 1 Parametrization S follows D 𝑢 𝐵 𝐾𝐸 David vid 1:10pm 𝐵 𝐾𝑇 𝑥 = 𝐵 𝐾𝐶 Sophie hie 𝐵 𝐾𝐷 B C 𝑂 𝐶 𝑢 𝑂 𝐷 𝑢 Terminating process adopt product only once 1:18pm 1:15pm Christine ine Bob Followee Not yet 𝐵 𝐾𝐷 𝐵 𝐾𝐶 adopted adopted 1:25pm J ℎ 𝐾∗ 𝑢 = 𝐵 𝐾𝐶 1 − 𝑂 𝐾 𝑢 𝑂 𝐶 𝑢 Jacob ob 𝑂 𝐾 𝑢 + 𝐵 𝐾𝐷 1 − 𝑂 𝐾 𝑢 𝑂 𝐷 𝑢 54

ℓ 1 Regularized log-likelihood 𝑛 𝑀 𝑥 + 𝜇 𝑥 1 = log 𝑥, 𝜚 ∗ 𝑢 𝑗 − 𝑥, Ψ ∗ 𝑈 − 𝜇 𝑥 1 𝑗=1 𝑈 time 𝑢 1 𝑢 3 𝑢 2 𝑢 Jacob ob Likelihood: ℎ ∗ 𝑢 1 ℎ ∗ 𝑢 2 ℎ ∗ 𝑢 3 ℎ ∗ 𝑢 exp − ℎ ∗ 𝜐 𝑒𝜐 𝑈 0 𝑥, 𝜚 ∗ 𝑢 1 𝑥, 𝜚 ∗ 𝑢 3 𝑈 𝑥, 𝜚 ∗ (𝜐) 𝑒𝜐 exp − 0 𝑥, 𝜚 ∗ 𝑢 2 𝑥, 𝜚 ∗ 𝑢 55

Soft-thresholding algorithm ℓ 1 -reguarlized likelihood estimation problem. Solve one such problem for each node. Set learning rate 𝛾 𝑙 = 0 Initialize 𝑥 While 𝑙 ≤ 𝐿 , do 𝑥 𝑙+1 = 𝑥 𝑙 − 𝛾 ⋅ 𝛼 𝑥 𝑀 𝑥 𝑙 − 𝜇 ⋅ 𝛾 + 𝑘 𝑘 𝑙 = 𝑙 + 1 End while 𝑣 𝑣 𝑗 𝑗 𝑤 𝑤 56

Statistical guarantees Recovery conditions: 2 𝑀, is bounded [𝐷 𝑛𝑗𝑜 , 𝐷 𝑛𝑏𝑦 ] Eigenvalue of the Hessian, 𝑅 = 𝛼 𝑥 Gradient is upper bounded, 𝛼 𝑥 𝑀 ∞ ≤ 𝐷 1 Hazard is lower bounded, min 𝑥 𝑘 ≥ 𝐷 2 Incoherence condition: 𝑅 𝑇 𝑑 𝑇 𝑅 𝑇𝑇 −1 ∞ ≤ 1 − 𝜁 network structure parameter value observation window source node distribution Given 𝑜 > 𝐷 3 ⋅ 𝑒 3 log 𝑞 cascades, set regularization parameter 2−𝜁 log 𝑞 𝜇 ≥ 𝐷 4 ⋅ 𝑜 , the network structure can be recovered with 𝜁 probability at least 1 − 2 exp(−𝐷 ′′ 𝜇 2 𝑜) 57

Memetracker 58

Estimated diffusion network Blogs Mainstream media Nan et al. NIPS 2012 59

Tracking diffusion networks 60

Dyn ynamic mic Pr Processes esses ove ver In Informat matio ion n Netwo works rks Rep epre rese sentat ntation, ion, Modeli deling ng, , Le Learning ning and d Infer erence ence Learning: Low Rank Collaborative Dynamics 61

Collaborative dynamics 𝑝 1 𝑝 2 Regularization 𝑝 3 𝑝 4 𝜈 𝐸𝑝 1 𝜈 𝐸𝑝 4 … ⋮ ⋱ ⋮ David id 𝜈 𝐾𝑝 1 𝜈 𝐾𝑝 4 … ∗ 𝛽 𝐸𝑝 1 𝛽 𝐸𝑝 4 Sophi phie … ⋮ ⋱ ⋮ 𝛽 𝐾𝑝 1 𝛽 𝐾𝑝 4 … ∗ Chris istine tine Self-exciting process Tend to go to the same store Jacob ob again and again ℎ 𝐸𝑝 1 ∗ 𝑢 = 𝜈 𝐸𝑝 1 + 𝛽 𝐸𝑝 1 𝐸𝑝 1 | exp −|𝑢 − 𝑢 𝑗 𝐸𝑝1 ∈𝓘 𝑢 𝐸𝑝1 𝑢 𝑗 62

Dyn ynamic mic Pr Processes esses ove ver In Informat matio ion n Netwo works rks Rep epre rese sentat ntation, ion, Modeli deling ng, , Le Learning ning and d Infer erence ence Learning: Generic Algorithm 63

Concave log-likelihood of event sequence Log-likelihood Concave in Concave in 𝑛 𝑀(𝑥) = log 𝑥, 𝜚 ∗ 𝑢 𝑗 w! w! − 𝑥, Ψ ∗ (𝑈) 𝑗=1 𝑈 time 𝑢 1 𝑢 3 𝑢 2 𝑢 Jacob ob Likelihood: ℎ ∗ 𝑢 1 ℎ ∗ 𝑢 2 ℎ ∗ 𝑢 3 ℎ ∗ 𝑢 exp − ℎ ∗ 𝜐 𝑒𝜐 𝑈 0 𝑥, 𝜚 ∗ 𝑢 1 𝑥, 𝜚 ∗ 𝑢 3 𝑈 𝑥, 𝜚 ∗ (𝜐) 𝑒𝜐 exp − 0 𝑥, 𝜚 ∗ 𝑢 2 𝑥, 𝜚 ∗ 𝑢 64

Challenge in optimization problem 𝑈 time David vid 𝑢 1 𝑢 2 𝑢 3 … 𝑢 𝑗 𝑢 𝑛 Negative log-likelihood 𝑛 𝑜 𝑥, Ψ ∗ (𝑈) − log 𝑥, 𝜚 ∗ (𝑢 𝑗 ) min + 𝜇 𝑥 1 w∈ℝ + 𝑗=1 Existing first order methods Existing first order methods log 𝑦 1 1 𝑃 𝑃 𝜗 2 iterations 𝜗 2 iterations Δ𝑧 Non-Lipschitz Δ𝑦 65

Saddle point reformulation 𝑈 time David vid 𝑢 1 𝑢 2 𝑢 3 … 𝑢 𝑗 𝑢 𝑛 Negative log-likelihood 𝑛 𝑜 𝑥, Ψ ∗ (𝑈) − log 𝑥, 𝜚 ∗ (𝑢 𝑗 ) min + 𝜇 𝑥 1 w∈ℝ + 𝑗=1 Fenchel dual 𝑤 𝑗 >0 𝑤 𝑗 𝑥, 𝜚 ∗ (𝑢 𝑗 ) − log 𝑤 𝑗 − 1 max 𝑛 𝑛 𝑥, Ψ ∗ (𝑈) − 𝑤 𝑗 𝑥, 𝜚 ∗ 𝑢 𝑗 min 𝑜 max + log 𝑤 𝑗 + 𝜇 𝑥 1 𝑛 w∈ℝ + 𝑤 𝑗 >0 𝑗=1 𝑗=1 𝑗=1 He et al. Arxiv 2016 66

Proximal gradient 𝑥 𝑤 𝑗 𝑘 (𝑥 𝑘 , 𝑤 𝑗 ) 𝑢 , 𝑤 𝑗 𝑢 ) (𝑥 𝑘 𝑢 − 𝛿 𝛼 𝑢 − 𝛿 𝛼 𝑥 𝑥 𝑘 = 𝑥 𝑘 = 𝑥 𝑘 − 𝜇𝛿 + 𝑘 − 𝜇𝛿 + 𝑢 𝑢 𝑥 𝑘 𝑀 𝑥 𝑢 , 𝑤 𝑗 𝑥 𝑘 𝑀 𝑥 𝑢 , 𝑤 𝑗 𝑥 𝑥 𝑘 = 𝑥 𝑘 = 𝑥 𝑘 𝑘 2 + 4𝛿 2 + 4𝛿 1/2 1/2 𝑗 = 𝑤 𝑗 + 𝑤 𝑗 𝑗 = 𝑤 𝑗 + 𝑤 𝑗 𝑢 + 𝛿 𝛼 𝑢 + 𝛿 𝛼 𝑢 𝑢 𝑤 𝑗 𝑀 𝑥 𝑢 , 𝑤 𝑗 𝑤 𝑗 𝑀 𝑥 𝑢 , 𝑤 𝑗 𝑤 𝑗 = 𝑤 𝑗 𝑤 𝑗 = 𝑤 𝑗 𝑤 𝑤 2 2 𝑢 } 𝑢 } Given current 𝑥 𝑢 , {𝑤 𝑗 Given current 𝑥 𝑢 , {𝑤 𝑗 𝑛 𝑛 𝑥, Ψ ∗ (𝑈) − 𝑤 𝑗 𝑥, 𝜚 ∗ 𝑢 𝑗 min 𝑜 max + log 𝑤 𝑗 + 𝜇 𝑥 1 𝑛 w∈ℝ + 𝑤 𝑗 >0 𝑗=1 𝑗=1 𝑗=1 Bilinear form 𝑀 𝑥, 𝑤 𝑗 67

Accelerated proximal gradient 𝑥 𝑤 𝑗 𝑘 𝑥 𝑢+1 = 𝑥 𝑥 𝑢+1 = 𝑥 𝑢+1 , 𝑤 𝑗 𝑢+1 ) 𝑢 − 𝛿 𝛼 𝑢 − 𝛿 𝛼 𝑘 − 𝜇𝛿 + 𝑘 − 𝜇𝛿 + (𝑥 𝑥 𝑥 𝑘 = 𝑥 𝑘 = 𝑥 𝑥 𝑘 𝑀 𝑥, 𝑤 𝑗 𝑥 𝑘 𝑀 𝑥 , 𝑤 𝑗 𝑘 𝑘 𝑘 2 + 4𝛿 2 + 4𝛿 1/2 1/2 (𝑥 𝑘 , 𝑤 𝑗 ) 𝑢+1 = 𝑤 𝑗 + 𝑤 𝑗 𝑢+1 = 𝑤 𝑗 + 𝑤 𝑗 𝑢 + 𝛿 𝛼 𝑢 + 𝛿 𝛼 𝑤 𝑗 𝑤 𝑗 𝑤 𝑗 = 𝑤 𝑗 𝑤 𝑗 = 𝑤 𝑗 𝑤 𝑗 𝑀 𝑥 𝑤 𝑗 𝑀 𝑥, 𝑤 𝑗 , 𝑤 𝑗 𝑢 , 𝑤 𝑗 𝑢 ) (𝑥 2 2 𝑘 𝑢 − 𝛿 𝛼 𝑢 − 𝛿 𝛼 𝑥 𝑥 𝑘 = 𝑥 𝑘 = 𝑥 𝑘 − 𝜇𝛿 + 𝑘 − 𝜇𝛿 + 𝑢 𝑢 𝑥 𝑘 𝑀 𝑥 𝑢 , 𝑤 𝑗 𝑥 𝑘 𝑀 𝑥 𝑢 , 𝑤 𝑗 𝑥 𝑥 𝑘 = 𝑥 𝑘 = 𝑥 𝑘 𝑘 2 + 4𝛿 2 + 4𝛿 1/2 1/2 𝑗 = 𝑤 𝑗 + 𝑤 𝑗 𝑗 = 𝑤 𝑗 + 𝑤 𝑗 𝑢 + 𝛿 𝛼 𝑢 + 𝛿 𝛼 𝑢 𝑢 𝑤 𝑗 𝑀 𝑥 𝑢 , 𝑤 𝑗 𝑤 𝑗 𝑀 𝑥 𝑢 , 𝑤 𝑗 𝑤 𝑗 = 𝑤 𝑗 𝑤 𝑗 = 𝑤 𝑗 𝑤 𝑤 2 2 1 1 𝑃 𝑃 𝜗 iterations 𝜗 iterations 𝑢 } 𝑢 } Given current 𝑥 𝑢 , {𝑤 𝑗 Given current 𝑥 𝑢 , {𝑤 𝑗 𝑛 𝑛 𝑥, Ψ ∗ (𝑈) − 𝑤 𝑗 𝑥, 𝜚 ∗ 𝑢 𝑗 min 𝑜 max + log 𝑤 𝑗 + 𝜇 𝑥 1 𝑛 w∈ℝ + 𝑤 𝑗 >0 𝑗=1 𝑗=1 𝑗=1 Bilinear form 𝑀 𝑥, 𝑤 𝑗 68

Converge much faster Accelerated gradient Unaccelerated 69

Dyn ynamic mic Pr Processes esses ove ver In Informat matio ion n Netwo works rks Rep epre rese sentat ntation, ion, Modeli deling ng, , Le Learning ning and d Infer erence ence Inference: Time-Sensitive Recommendation 70

Collaborative dynamics 𝑝 1 𝑝 2 𝑝 3 𝑝 4 David vid Sophie hie Next item prediction Next item prediction What next item David will buy? What next item David will buy? ℎ 𝐸𝑝∗ (𝑢) ℎ 𝐸𝑝∗ (𝑢) max max 𝑝 𝑝 Christine ine Return time prediction Return time prediction When will David buy the item? When will David buy the item? ∞ ∞ Jacob ob max 𝜐𝑔 𝐸𝑝∗ 𝜐 𝑒𝜐 max 𝜐𝑔 𝐸𝑝∗ 𝜐 𝑒𝜐 𝑢 𝑢 Nan et al. NIPS 2015 71

Music recommendation for Last.fm Online records of music listening. The time unit is hour 1000 users, 3000 albums 20,000 observed pairs, more than 1 million events Album prediction Returning time prediction 72

Electronic healthcare records MIMIC II dataset: a collection of de-identified clinical visit records The time unit is week 650 patients and 204 disease codes Diagnosis code prediction Returning time prediction 73

Dyn ynamic mic Pr Processes esses ove ver In Informat matio ion n Netwo works rks Rep epre rese sentat ntation, ion, Modeli deling ng, , Le Learning ning and d Infer erence ence Inference: Influence Maximization 74

Inference in dea adoption D S means 1:18 pm Influence estimation Influence estimation S follows D … Can a piece of news spread, in 1 month, Can a piece of news spread, in 1 month, David vid to a million user? to a million user? 1:30 pm 𝜏 𝑡, 𝑢 : = 𝔽 𝑂 𝑗 (𝑢) 𝜏 𝑡, 𝑢 : = 𝔽 𝑂 𝑗 (𝑢) … Sophie hie 𝑗∈𝑊 𝑗∈𝑊 Influence maximization Influence maximization Who is the most influential user? Who is the most influential user? 2:00 pm max max 𝑡∈𝑊 𝜏 𝑡, 𝑢 𝑡∈𝑊 𝜏 𝑡, 𝑢 … Christine ine Bob Source localization Source localization Where is the origin of information? Where is the origin of information? 𝑡∈𝑊,𝑢∈[0,𝑈] Likelihood partial cascade 𝑡∈𝑊,𝑢∈[0,𝑈] Likelihood partial cascade max max Jacob ob Rodriguez et al. ICML 2012 Nan et al. NIPS 2013 75 Farajtabar et al. AISTATS 2015

Cascades from D in 1 month David vid David vid David vid Sophie hie Sophie hie Sophie hie Bob Christine ine Jacob ob Bob Christine ine Bob Christine ine Jacob ob 76 Jacob ob

Cascades from D in 1 month 𝑂 𝑗 𝑢 = 4 𝑂 𝑗 𝑢 = 2 𝑂 𝑗 𝑢 = 3 𝑗∈𝑊 𝑗∈𝑊 𝑗∈𝑊 David vid David vid David vid Sophie hie Sophie hie Sophie hie Bob Christine ine Jacob ob 𝜏 𝐸, 𝑢 Bob ≈ 4 + 2 + 3 Christine ine Bob Christine ine 3 = 3 Jacob ob 77 Jacob ob

Cascades from B in 1 month David vid David vid David vid Sophie hie Sophie hie Sophie hie Bob Christine ine Jacob ob Bob Christine ine Bob Christine ine Jacob ob 78 Jacob ob

Cascades from B in 1 month 𝑂 𝑗 𝑢 = 4 𝑂 𝑗 𝑢 = 2 𝑂 𝑗 𝑢 = 2 𝑗∈𝑊 𝑗∈𝑊 𝑗∈𝑊 David vid David vid David vid Sophie hie Sophie hie Sophie hie Bob Christine ine Jacob ob 𝜏 𝐶, 𝑢 Bob ≈ 4 + 2 + 2 Christine ine Bob Christine ine 3 = 2.67 Jacob ob 79 Jacob ob

Find most influential user max max 𝑡∈𝑊 𝜏 𝑡, 𝑢 𝑡∈𝑊 𝜏 𝑡, 𝑢 𝑃 𝑞 𝑊 𝑊 + |𝐹| 𝑃 𝑛 𝑊 2 + 𝑛 𝐹 |𝑊| 𝑛 𝑃 𝑊 2 + 𝐹 |𝑊| Each graph 𝑃 David vid David vid David vid Soph phie ie Sophie phie Soph phie ie Bob Bob Chris istin tine Jacob cob Bob Bob Chris istin tine Bob Bob Chris istin tine 80 Jacob cob Jacob cob

Find most influential user max max 𝑡∈𝑊 𝜏 𝑡, 𝑢 𝑡∈𝑊 𝜏 𝑡, 𝑢 𝑃 𝑞 𝑊 𝑊 + |𝐹| 𝑃 𝑛 𝑊 2 + 𝑛 𝐹 |𝑊| 𝑛 𝑃 𝑊 2 + 𝐹 |𝑊| Each graph Each node 𝑃 David vid David vid David vid Soph phie ie Sophie phie Soph phie ie Bob Bob Chris istin tine Jacob cob Bob Bob Chris istin tine Bob Bob Chris istin tine 81 Jacob cob Jacob cob

Find most influential user Quadratic in |𝑊| Quadratic in |𝑊| max max 𝑡∈𝑊 𝜏 𝑡, 𝑢 𝑡∈𝑊 𝜏 𝑡, 𝑢 𝑃 𝑞 𝑊 𝑊 + |𝐹| not scalable! not scalable! 𝑃 𝑛 𝑊 2 + 𝑛 𝐹 |𝑊| 𝑛 𝑃 𝑊 2 + 𝐹 |𝑊| Each graph Each node Single source shortest path 𝑃 David vid David vid David vid Soph phie ie Sophie phie Soph phie ie Bob Bob Chris istin tine Jacob cob Bob Bob Chris istin tine Bob Bob Chris istin tine 82 Jacob cob Jacob cob

Randomized neighborhood estimation 𝑠 ∼ exp(−𝑠) Linear in # of Linear in # of nodes and edges nodes and edges David vid 2.75 𝑆 𝐸 = 0.29 Sophie hie 1.38 David id 0.29 𝑆 𝑇 = 0.29 Bob Sophi phie 𝑆 𝐶 = 0.29 Bob Jacob ob 1.26 𝑆 𝐾 = 1.26 Jacob ob 𝑆 𝐷 = 0.33 0.33 Christine ine Chris istine tine 83

Randomized neighborhood estimation 𝑛 − 1 𝑛 − 1 𝑠 ∼ exp(−𝑠) 𝜏 𝑡, 𝑢 ≈ 𝜏 𝑡, 𝑢 ≈ 𝑛 𝑛 𝑆 𝑡 (𝑗) 𝑆 𝑡 (𝑗) 𝑗=1 𝑗=1 David vid 0.23 𝑆 𝐸 = 0.29, 0.23 Sophie hie 0.32 David id 1.97 𝑆 𝑇 = 0.29, 0.23 Bob Sophi phie 𝑆 𝐶 = 0.29, 0.23 Bob Jacob ob 0.37 𝑆 𝐾 = 1.26, 0.37 Given 𝑛 iid samples, 𝑠 ∼ 𝑓 −𝑠 , Given 𝑛 iid samples, 𝑠 ∼ 𝑓 −𝑠 , Jacob ob their minimum 𝑠 their minimum 𝑠 ∗ is distributed as ∗ is distributed as 𝑆 𝐷 = 0.33, 3.70 ∗ ∼ 𝑛𝑓 −𝑛𝑠 ∗ ∼ 𝑛𝑓 −𝑛𝑠 𝑠 𝑠 3.70 Christine ine Chris istine tine 84

Computational complexity 𝑞 𝑞 𝑃 𝑞 𝑛 𝑊 + 𝑊 + 𝐹 𝜏 𝑡, 𝑢 ≈ 1 𝜏 𝑡, 𝑢 ≈ 1 𝑛 − 1 𝑛 − 1 𝑞 𝑞 𝑛 𝑛 𝑡 (𝑗) 𝑡 (𝑗) 𝑆 𝑆 𝑗=1 𝑗=1 𝑘 𝑘 𝑘=1 𝑘=1 Each graph Each random Each node Breadth first label set search David vid David vid David vid Soph phie ie Sophie phie Soph phie ie Bob Bob Chris istin tine Jacob cob Bob Bob Chris istin tine Bob Bob Chris istin tine 85 Jacob cob Jacob cob

Scalability 86

Ten most influential sites in a month Site Typ ype e of site digg.com popular news site lxer.com linux and open source news exopolitics.blogs.com political blog mac.softpedia.com mac news and rumors gettheflick.blogspot.com pictures blog urbanplanet.org urban enthusiasts givemeaning.blogspot.com political blog talkgreen.ca environmental protection blog curriki.org educational site pcworld.com technology news 87

Dyn ynamic mic Pr Processes esses ove ver In Informat matio ion n Netwo works rks Rep epre rese sentat ntation, ion, Modeli deling ng, , Le Learning ning and d Infer erence ence More Advanced Models 88

Nan et al. AISTATS 203 Joint models with rich context Nan et al. KDD 2015 Audio Text time 0 𝑢 1 𝑢 2 … … 𝑢 𝑛 𝑢 𝑗 𝑈 Image Other simultaneously measured time-series 89

Spatial temporal processes influenza spread bird migration Crime Smart city 90

Continuous-time document streams Time Nan et al. KDD 2015 91

Dirichlet-Hawkes processes Dirichlet Recurrent Hawkes Chinese Restaurant Process Process ℎ 𝑙 (𝑢 𝑜 ) 𝛽 𝜄 𝑜 |𝜄 1:𝑜−1 ∼ + 𝛽 𝜀 𝜄 𝑙 + + 𝛽 𝐻 0 𝜄 ℎ 𝑙 ′ (𝑢 𝑜 ) ℎ 𝑙 ′ (𝑢 𝑜 ) 𝑙 ′ 𝑙 ′ 𝑙 92

Dark Knight vs. Endeavour Triggering Kernel Temporal Dynamics 93

Previous models are parametric Each parametric form encodes our prior knowledge Poisson Process Hawkes Process Self-Correcting Process Autoregressive Conditional Duration Process Limitations Model may be misspecified Hard to encode complex features or markers Hard to encode dependence structure Can we Ca we learn a mo more exp xpressive ssive mo mode del of ma marked ked temp mporal al po point t pr processes esses ? 94

Recurrent Marked Temporal Point Processes Recurrent neural network + Marked temporal point processes hidden vector of RNN learns a nonlinear dependency over both past ti time and marker ers general conditional density multinomial distribution of the next timing for the markers 95

Experiments: synthetic Time Prediction Intensity Function Prediction Error ACD Hawkes Self-Correcting 96

Experiments: real world data NYC Taxi Trading Stackoverflow MIMIC-II Time Prediction Marker Prediction 97

A unified framework Representation P ROBABILISTIC M ODELS 1. Intensity function and 2. Basic building blocks L EARNING M ETHODS 3. Superposition to Modeling 1. Idea adoption understand 2. Network coevolution predict 3. Collaborative dynamics control Learning 1. Sparse hidden diffusion networks P ROCESSES & A CTIVITY 2. Low rank collaborative dynamics over 3. Generic algorithm S OCIAL & I NFORMATION Inference N ETWORKS 1. Time-sensitive recommendation 2. Scalable Influence estimation 98

Introduction to PtPack A C++ ++ Mu Multivariate variate Tem empor oral al Poi oint nt Proce ocess ss Packa ackage ge

Features Learning sparse interdependency structure of continuous-time information diffusions Scalable continuous-time influence estimation and maximization Learning multivariate Hawkes processes with different structural constraints, like: sparse, low-rank, customized triggering kernels Learning low-rank Hawkes processes for time-sensitive recommendations Efficient simulation of standard multivariate Hawkes processes Learning multivariate self-correcting processes Simulation of customized general temporal point processes Basic residual analysis and model checking of customized temporal point processes Visualization of triggering kernels, intensity functions, and simulated events 100