the graph hawkes neural network for forecasting on
play

The Graph Hawkes Neural Network for Forecasting on Temporal - PowerPoint PPT Presentation

The Graph Hawkes Neural Network for Forecasting on Temporal Knowledge Graphs By Zhen Han, Yunpu Ma, Yuyi Wang, Stephan Gnnemann, Volker Tresp Page 1 Zhen Han, The Graph Hawkes Neural Network for Forecasting on Temporal Knowledge Graphs, AKBC


  1. The Graph Hawkes Neural Network for Forecasting on Temporal Knowledge Graphs By Zhen Han, Yunpu Ma, Yuyi Wang, Stephan Günnemann, Volker Tresp Page 1 Zhen Han, The Graph Hawkes Neural Network for Forecasting on Temporal Knowledge Graphs, AKBC 2020, June 22 th .

  2. Temporal Knowledge Graph (tKG) Each Quadruple represents an events : (subject, predicate, object, timestamp) (Obama, visit, Turkey, 2009-04-05) Global Database of Events, Language, and Tone (GDELT) Page 2 Zhen Han, The Graph Hawkes Neural Network for Forecasting on Temporal Knowledge Graphs, AKBC 2020, June 22 th .

  3. Graph View of a Temporal Knowledge Graph Each Quadruple represents an events : (subject, predicate, object, timestamp) (Obama, visit, Turkey, 2009-04-05) Page 3 Zhen Han, The Graph Hawkes Neural Network for Forecasting on Temporal Knowledge Graphs, AKBC 2020, June 22 th .

  4. From Temporal Knowledge Graph to Event Sequence 6 t 1 2 5 p 3 1 p 2 p 1 p 1 3 4 p 3 (e 1, p 1 , e 2 ), (e 1, p 2, e 5 ), (e 6, p 3, e 1 ), (e 3, p 2, e 2 ), (e 3, p 3, e 4 ) t 1 t Timeline of a Sequence of Events. Page 4 Zhen Han, The Graph Hawkes Neural Network for Forecasting on Temporal Knowledge Graphs, AKBC 2020, June 22 th .

  5. From Temporal Knowledge Graph to Event Sequence 6 6 t 1 t 2 2 5 2 5 p 3 p 3 1 1 p 1 p 2 p 2 p 1 p 1 p 1 p 1 3 4 3 4 p 3 (e 1, p 1 , e 2 ), (e 1, p 2, e 5 ), (e 6, p 3, e 1 ), (e 3, p 2, e 2 ), (e 3, p 3, e 4 ) (e 1, p 1 , e 3 ), (e 1, p 2, e 4 ), (e 1, p 2, e 5 ), (e 6, p 3, e 1 ), (e 3, p 1, e 2 ) t 1 t 2 t Timeline of a Sequence of Events. Page 5 Zhen Han, The Graph Hawkes Neural Network for Forecasting on Temporal Knowledge Graphs, AKBC 2020, June 22 th .

  6. From Temporal Knowledge Graph to Event Sequence 6 6 6 t 1 t 2 t 3 p 3 2 5 2 5 2 5 p 3 p 3 p 2 1 1 1 p 1 p 2 p 2 p 2 p 1 p 1 p 1 p 1 p 2 p 1 3 4 3 4 3 4 p 3 Slices of a Discrete-time Temporal Knowledge Graph. (e 1, p 1 , e 2 ), (e 1, p 2, e 5 ), (e 6, p 3, e 1 ), (e 3, p 2, e 2 ), (e 3, p 3, e 4 ) (e 1, p 1 , e 3 ), (e 1, p 2, e 4 ), (e 1, p 2, e 5 ), (e 6, p 3, e 2 ), (e 3, p 2, e 2 ) (e 1, p 1 , e 3 ), (e 1, p 2, e 4 ), (e 1, p 2, e 5 ), (e 6, p 3, e 1 ), (e 3, p 1, e 2 ) t 1 t 2 t 3 t Timeline of a Sequence of Events. Page 6 Zhen Han, The Graph Hawkes Neural Network for Forecasting on Temporal Knowledge Graphs, AKBC 2020, June 22 th .

  7. From Temporal Knowledge Graph to Event Sequence 6 6 6 6 t 1 t 2 t 3 t 4 ? p 3 p 3 p 3 2 5 2 5 2 5 2 5 p 3 p 3 p 2 1 1 1 1 p 1 p 2 2 p p 2 p 1 p 2 p 1 p 1 p 1 p 1 p 2 p 2 p 2 p 1 3 4 3 4 3 4 3 4 p 3 Slices of a Temporal Knowledge Graph. (e 1, p 1 , e 2 ), (e 1, p 2, e 5 ), (e 6, p 3, e 1 ), (e 3, p 2, e 2 ), (e 3, p 3, e 4 ) (e 1, p 1 , e 3 ), (e 1, p 2, e 4 ), (e 1, p 2, e 5 ), (e 6, p 3, e 2 ), (e 3, p 2, e 2 ) (e 1, p 1 , e 3 ), (e 1, p 2, e 4 ), (e 1, p 2, e 5 ), (e 6, p 3, e 1 ), (e 3, p 1, e 2 ) t 1 t 2 t 3 t 4 Timeline of a Sequence of Events. Page 7 Zhen Han, The Graph Hawkes Neural Network for Forecasting on Temporal Knowledge Graphs, AKBC 2020, June 22 th .

  8. Hawkes Process & Neural Hawkes Process 𝜇 𝑙 𝑢 = 𝜈 ! + ∑ ":$ ! %$ 𝛽 ! ! ,! exp −𝜀 ! ! ,! 𝑢 − 𝑢 " Hawkes Process [2] . Exponential decaying with time Intensity function of event type 𝑙 Mutual excitation Base intensity Page 8 Zhen Han, The Graph Hawkes Neural Network for Forecasting on Temporal Knowledge Graphs, AKBC 2020, June 22 th .

  9. Hawkes Process & Neural Hawkes Process 𝜇 𝑙 𝑢 = 𝜈 ! + ∑ ":$ ! %$ 𝛽 ! ! ,! exp −𝜀 ! ! ,! 𝑢 − 𝑢 " Hawkes Process [2] . Exponential decaying with time Intensity function of event type 𝑙 Mutual excitation Base intensity ' 𝒊(𝑢)) . 𝜇 ! 𝑢 = 𝑔(𝒙 ! Neural Hawkes Process [3] Intensity function of event type 𝑙 Activation function Event-specific weight vector Hidden state vector Intensity-2 Intensity-1 Excitation BaseIntensity-2 Inhibition BaseIntensity-1 t EventType-1 EventType-2 EventType-1 EventType-2 t 1 t 2 t 3 t 4 An Event Stream from the Neural Hawkes Process. Page 9 Zhen Han, The Graph Hawkes Neural Network for Forecasting on Temporal Knowledge Graphs, AKBC 2020, June 22 th .

  10. Challenge: Characteristics of Temporal Knowledge Graphs • Scalability: a huge amount of event types in tKGs. Number of probable event types in our tKG dataset: 1.4 ⋅ 10 10 o (subject, predicate, object) Existing event types in our dataset: 1.2 ⋅ 10 6 o (e 1, p 1 , e 2 ), (e 1, p 2, e 5 ), (e 6, p 3, e 1 ), (e 3, p 2, e 2 ), (e 3, p 3, e 4 ) (e 1, p 1 , e 3 ), (e 1, p 2, e 4 ), (e 1, p 2, e 5 ), (e 6, p 3, e 2 ), (e 3, p 2, e 2 ) (e 1, p 1 , e 3 ), (e 1, p 2, e 4 ), (e 1, p 2, e 5 ), (e 6, p 3, e 1 ), (e 3, p 2, e 2 ) t 1 t 2 t 3 t Event Sequence Extracted from a Temporal Knowledge Graph Page 10 Zhen Han, The Graph Hawkes Neural Network for Forecasting on Temporal Knowledge Graphs, AKBC 2020, June 22 th .

  11. How to improve the scalability of Hawkes process? • Considering an object prediction query (e 1, p 1, ? , t 4 ). Page 11 Zhen Han, The Graph Hawkes Neural Network for Forecasting on Temporal Knowledge Graphs, AKBC 2020, June 22 th .

  12. How to improve the scalability of Hawkes process? • Considering an object prediction query (e 1, p 1, ? , t 4 ). • Modelling intensity functions inspired by score functions of KGs Page 12 Zhen Han, The Graph Hawkes Neural Network for Forecasting on Temporal Knowledge Graphs, AKBC 2020, June 22 th .

  13. How to improve the scalability of Hawkes process? • Considering an object prediction query (e 1, p 1, ? , t 4 ). • Modelling intensity functions inspired by score functions of KGs • Investigating the influence of the following historical event sequence: e h,sp (e 1, p 1, t 4 ) = {(e 1, p 1 , e 3, t 1 ), (e 1, p 1 , e 4, t 1 ), (e 1, p 1 , e 2, t 2 ), (e 1, p 1 , e 4, t 2 ), (e 1, p 1, e 3, t 3 )}. (e 1, p 1 , e 3 ), (e 1, p 1, e 4 ), (e 6, p 3, e 1 ), (e 3, p 2, e 2 ), (e 3, p 3, e 4 ) (e 1, p 1 , e 3 ), (e 1, p 2, e 4 ), (e 1, p 2, e 5 ), (e 6, p 3, e 2 ), (e 3, p 2, e 2 ) (e 1, p 1 , e 2 ), (e 1, p 1, e 4 ), (e 1, p 2, e 5 ), (e 6, p 3, e 1 ), (e 3, p 1, e 2 ) t 1 t 2 t 3 t Event Sequence Extracted from a Temporal Knowledge Graph Page 13 Zhen Han, The Graph Hawkes Neural Network for Forecasting on Temporal Knowledge Graphs, AKBC 2020, June 22 th .

  14. Neighborhood Aggregation • Considering an object prediction query (e 1, p 1, ? , t 4 ). g(O t1 (e 1 , p 2 )) • Neighborhood Aggregation Module [1] : t 1 e 1 p 1 1 p O t1 (e 1 , p 2 ) e 3 1 e 4 g O " ! e # , 𝑞 $ = (𝐟 % + 𝐟 & ) O " ! e # , p $ Neighborhood Aggregation ={e 3 , e 4 } Embedding of the 3-th entity Embedding of the 4-th entity (e 1, p 1 , e 3 ), (e 1, p 1, e 4 ), (e 6, p 3, e 1 ), (e 3, p 2, e 2 ), (e 3, p 3, e 4 ) (e 1, p 1 , e 3 ), (e 1, p 2, e 4 ), (e 1, p 2, e 5 ), (e 6, p 3, e 2 ), (e 3, p 2, e 2 ) (e 1, p 1 , e 2 ), (e 1, p 1, e 4 ), (e 1, p 2, e 5 ), (e 6, p 3, e 1 ), (e 3, p 1, e 2 ) t 1 t 2 t 3 t Event Sequence Extracted from a Temporal Knowledge Graph Page 14 Zhen Han, The Graph Hawkes Neural Network for Forecasting on Temporal Knowledge Graphs, AKBC 2020, June 22 th .

  15. Graph Hawkes Process • Object prediction query e ' " , e ( " , ? , t ) . • Hidden state computed by a continuous-time LSTM (cLSTM) network [3] ,,'( ) 𝐢 '*+ e ' " , e ( " , t ) , e ) = cLSTM 𝐟 ' " , 𝐟 ( " , ∪ ./# g O " # (e ' " , e ( " ) Historical event sequence Subject embedding Predicate embedding Neighborhood aggregation module Page 15 Zhen Han, The Graph Hawkes Neural Network for Forecasting on Temporal Knowledge Graphs, AKBC 2020, June 22 th .

  16. Graph Hawkes Process • Object prediction query e ' " , e ( " , ? , t ) . • Hidden state computed by a continuous-time LSTM (cLSTM) network [3] ,,'( ) 𝐢 '*+ e ' " , e ( " , t ) , e ) = cLSTM 𝐟 ' " , 𝐟 ( " , ∪ ./# g O " # (e ' " , e ( " ) Historical event sequence Subject embedding Predicate embedding Neighborhood aggregation module Inner product • Subject-centric intensity function ,,'( ⊕ 𝐟 ( " > 𝐟 0 ,,'( λ '*+ e 0 |e ' " , e ( " , t ) , e ) = f 𝐗 1 𝐟 ' " ⊕ 𝐗 , 𝐢 '*+ e ' " , e ( " , t ) , e ) Historical event sequence Subject embedding Hidden state vector Object embedding Predicate embedding Page 16 Zhen Han, The Graph Hawkes Neural Network for Forecasting on Temporal Knowledge Graphs, AKBC 2020, June 22 th .

  17. Link Prediction Task 3,45 . • Consider an object prediction query e ' " , e ( " , ? , t ) and the corresponding 𝑓 2 • Choose the object candidate with the highest intensity. Candidates Rank (,!" ) λ #$% (e ! ! , e " ! , e & , t ' , e ' e 1 1 (,!" ) e 2 λ #$% (e ! ! , e " ! , e * , t ' , e ' Query 2 e ! ! e " ! t i (,!" ) e 3 λ #$% (e ! ! , e " ! , e + , t ' , e ' 3 (,!" ) e 4 λ #$% (e ! ! , e " ! , e , , t ' , e ' 4 (,!" ) e N λ #$% (e ! ! , e " ! , e - , t ' , e ' N Page 17 Zhen Han, The Graph Hawkes Neural Network for Forecasting on Temporal Knowledge Graphs, AKBC 2020, June 22 th .

  18. Time Prediction Task • Given a time prediction query e ' " , e ( " , e 0 " , t = ? for t > t L. Last occurrence time of the given event type Page 18 Zhen Han, The Graph Hawkes Neural Network for Forecasting on Temporal Knowledge Graphs, AKBC 2020, June 22 th .

Recommend


More recommend