A Recurrent Neural Cascade-based Model for Continuous-Time Diffusion - PowerPoint PPT Presentation

A Recurrent Neural Cascade-based Model for Continuous-Time Diffusion Sylvain Lamprier LIP6 - Sorbonne Universit´ es 1 / 5

Cascade-based models for diffusion Information spreads from users to users in the network, following independent transmission probabilities 0.6 0.4 B E 0.2 0.1 0.7 D A 1 0.6 0.3 C F Observed Diffusion Episode = {(A;1);(B;2);(C;2);(D;3);(F;4)} The Continuous-Time Independent Cascade Model (CTIC) defines two parameters k u , v and r u , v per pair ( u , v ) of nodes in the network (Saito et al., 2011) : k u , v : probability that u succeeds in infecting v ; r u , v : time-delay parameter from u to v Likelihood of a set of episodes D : � � � P (v infected at time t D P ( D ) = v ) P ( v not infected in D ) D ∈D v ∈ U D v �∈ U D 2 / 5

RNN models for diffusion Markovian assumption does not hold in many situations : T ype 1 T ype 2 A D A D C C B E B E High proba for D High proba for E if A is infected if B is infected ⇒ Episode D as a sequence (( U D 1 , t D 1 ) , ( U D 2 , t D 2 ) , ..., ( U D | D | , t D | D | )) Recurrent Marked Temporal Point Processes (Du et al, 2016) : D D D D D D D D P(stop|h |D| ) P(U 1 |h 0 ) P(t 1 |h 0 ) P(U 2 |h 1 ) P(t 2 -t 1 |h 1 ) P(U 3 |h 2 ) P(t 3 -t 2 |h 2 ) hidden h 0 hidden h 1 hidden h 2 hidden h |D| time D D D D D D (U 1 ,t 1 ) (U 2 ,t 2 ) (U |D| ,t |D| ) ... But diffusion is not a sequence ! 3 / 5

RNN models for diffusion Tree Dependencies D F C A B E time D t 3 D D D D D t 1 t 2 t 4 t 5 t 6 F does not depend on E ⇒ Cyan (Wang et al., 2017b) : RNN with attention to select previous states ⇒ DAN (Wang et al., 2018) : Similar to Cyan, but with a pooling mechanism rather than RNN 4 / 5

Hybrid Recurrent / Cascade-Based Model for Diffusion v ∈ R d to each infected ⇒ Idea : Assign a continuous state z D node v , which depends on its infection path z D v then conditions distributions of subsequent infections from v � � u , ω ( k ) , with ω ( k ) ∈ R d a continuous < z D P( u infects v )= σ > v v representation of v If u is the first node to infect v : u , ω ( f ) z D v = f φ ( z D v ) with : f φ a GRU cell z D u the state of u for D (the memory) ∈ R d a static representation for v (the input) ω ( f ) v 5 / 5

Hybrid Recurrent / Cascade-Based Model for Diffusion v ∈ R d to each infected ⇒ Idea : Assign a continuous state z D node v , which depends on its infection path z D v then conditions distributions of subsequent infections from v 1 A 4 C 2 B D 6 5 P(infection from J) E 3 F 7 G K L M O I H 8 9 J 10 K O L M 5 / 5

Hybrid Recurrent / Cascade-Based Model for Diffusion v ∈ R d to each infected ⇒ Idea : Assign a continuous state z D node v , which depends on its infection path z D v then conditions distributions of subsequent infections from v 1 A 4 C 2 B D 6 5 P(infection from J) E 3 F 7 G K L M O I H 8 9 J 10 K O L M 5 / 5

Hybrid Recurrent / Cascade-Based Model for Diffusion v ∈ R d to each infected ⇒ Idea : Assign a continuous state z D node v , which depends on its infection path z D v then conditions distributions of subsequent infections from v 1 A 4 C 2 B D 6 5 P(infection from J) E 3 F 7 G K L M O I H 8 9 J 10 K O L M 5 / 5

Hybrid Recurrent / Cascade-Based Model for Diffusion v ∈ R d to each infected ⇒ Idea : Assign a continuous state z D node v , which depends on its infection path z D v then conditions distributions of subsequent infections from v 1 A 4 C 2 B D 6 5 P(infection from J) E 3 F 7 G K L M O I H 8 9 J 10 K O L M 5 / 5

Hybrid Recurrent / Cascade-Based Model for Diffusion v ∈ R d to each infected ⇒ Idea : Assign a continuous state z D node v , which depends on its infection path z D v then conditions distributions of subsequent infections from v Inference on ancestors sequences I is required : � log p ( D ) = log p ( D , I ) I ∈I D | D |− 1 � Inference distribution : q D ( I ) = p ( I i | D ≤ i , I < i ) ≈ p ( I | D ) i =1 Score function estimator : ∇ Θ L ( D ; Θ) =     � � � log p I ( D ) − b ∇ Θ log q D ( I ) + ∇ Θ log p I ( D )   E I ∼ q D    � �� �  � �� � D ∈D increases likelihood favors good paths given the path 5 / 5