http://cs224w.stanford.edu Evolving Networks are networks that - PowerPoint PPT Presentation

CS224W: Machine Learning with Graphs Jure Leskovec and Baharan Mirzasoleiman, Stanford http://cs224w.stanford.edu

¡ Evolving Networks are networks that change as a function of time ¡ Almost all real world networks evolve either by adding or removing nodes or links over time ¡ Examples : § Social networks : people make and lose friends and join or leave the network § Internet, web graphs, E-mail, phone calls, P2P networks, etc. Collaborations in the journal Physical Review Letters (PRL) [Perra et al. 2012] 11/14/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 2

¡ Visualization of the student collaboration network Nodes represent the students. An edge exists between two nodes if any of the two ever reported collaboration with the other in any of the assignments used to construct the network [Burstein et al. 2018] 11/14/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 3

¡ Evolution of the pooled R&D network for the nodes belonging to the ten largest sectors [Tomasello et al. 2017] 11/14/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 4

¡ Evolution of five selected sectoral R&D networks Blue nodes represent the firms strictly belonging to the examined sector, while orange nodes [Tomasello et al. 2017] represent their alliance partners belonging to different sectors 11/14/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 5

¡ Evolving network structure of academic institutions Community structure, indicated by color, for the networks from the three years 2011 to 2013. Different communities are indicated by different colors. [Wang et al. 2017] 11/14/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 6

¡ The largest components in Apple’s inventor network over a 6- year period Each node reflects an inventor, each tie reflects a patent collaboration. Node colors reflect technology classes, while node sizes show the overall connectedness of an inventor by measuring their total number of ties/collaborations (the node’s so-called degree centrality ). [kenedict.com] 11/14/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 7

¡ How do networks evolve? § How do networks evolve at the macro level? § Evolving network models, densification § How do networks evolve at the meso level? § Network motifs, communities § How do networks evolve at the micro level? § Node, link properties (degree, network centrality) Microscopic: Mesoscopic: Macroscopic: Degree, centralities Motifs, communities statistics 11/14/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 8

¡ How do networks evolve at the macro level? § What are global phenomena of network growth? ¡ Questions: § What is the relation between the number of nodes n(t) and number of edges e(t) over time t ? § How does diameter change as the network grows? § How does degree distribution evolve as the network grows? 11/14/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 10

¡ 𝑶(𝒖) … nodes at time 𝒖 ¡ 𝑭(𝒖) … edges at time 𝒖 ¡ Suppose that 𝑶 𝒖 + 𝟐 = 𝟑 ⋅ 𝑶(𝒖) ¡ Q: what is: 𝑭 𝒖 + 𝟐 = ? Is it 𝟑 ⋅ 𝑭(𝒖) ? ¡ A: More than doubled! § But obeying the Densification Power Law 11/14/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 11

¡ What is the relation between Internet the number of nodes and the E(t) edges over time? a=1.2 ¡ First guess: constant average degree over time N(t) ¡ Networks become denser over time Citations ¡ Densification Power Law: E(t) a=1.6 a … densification exponent (1 ≤ a ≤ 2) N(t) 11/14/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 12

¡ Densification Power Law § the number of edges grows faster than the number of nodes – average degree is increasing or equivalently a … densification exponent: 1 ≤ a ≤ 2: § a=1: linear growth – constant out-degree (traditionally assumed) § a=2: quadratic growth – fully connected graph 11/14/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 13

¡ Prior models and intuition say Internet that the network diameter slowly diameter grows (like log N) size of the graph Citations ¡ Diameter shrinks over time diameter § As the network grows the distances between the nodes slowly decrease How do we compute diameter in practice? time -- Long paths: Take 90 th -percentile or average path length (not the maximum) -- Disconnected components: Take only largest component or average only over connected pairs of nodes 11/14/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 14

Is shrinking Erdos-Renyi diameter just a random graph diameter consequence of densification? Densification (answer by exponent a =1.3 simulation) size of the graph Densifying random graph has increasing diameter Þ There is more to shrinking diameter than just densification! 11/14/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 15

Does the changing degree sequence explain the Citations shrinking diameter? diameter Compare diameter of a: § Real network ( red ) § Random network with the same degree distribution ( blue ) year Densification + degree sequence gives shrinking diameter 11/14/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 16

¡ How does degree distribution evolve to allow for densification? ¡ Option 1) Degree exponent 𝜹 𝒖 is constant: § Fact 1: If 𝜹 𝒖 = 𝜹 ∈ [𝟐, 𝟑] , then: 𝒃 = 𝟑/𝜹 Email network ■ Power-laws with exponents <2 have infinite expectations. ■ So, by maintaining constant degree exponent 𝛽 the average degree grows. 11/14/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 17

¡ How does degree distribution evolve to allow for densification? ¡ Option 2) 𝜹 𝒖 evolves with graph size 𝒐 : 𝒚7𝟐 8𝟐 𝟓𝒐 𝒖 § Fact 2: If 𝜹 𝒖 = 𝒚7𝟐 8𝟐 , then: 𝒃 = 𝒚 Notice: 𝜹 < → 2 𝟑𝒐 𝒖 as 𝑜 < → ∞ Citation network Remember, the expected degree in a power law is: 𝑭 𝒀 = 𝜹 𝒖 − 𝟐 𝜹 𝒖 − 𝟑 𝒚 𝒏 So 𝜹 𝒖 has to decay as a function of graph size 𝒐 𝒖 for the avg. degree to go up. 11/14/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 18

¡ Want to model graphs that densify and have shrinking diameters ¡ Intuition: § How do we meet friends at a party? § How do we identify references when writing papers? w v 11/14/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 19

¡ The Forest Fire model has 2 parameters: § p … forward burning probability § r … backward burning probability ¡ The model: Directed Graph § Each turn a new node v arrives § Uniformly at random choose an “ambassador” node w § Flip 2 coins sampled from a geometric distribution (based on p and r ) to determine the number of in- and out-links of w to follow, i.e., to ”spread the fire” along § “Fire” spreads recursively until it dies § New node v links to all burned nodes Geometric distribution: 11/14/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 20

¡ The Forest Fire model § (1) 𝑤 chooses an ambassador node 𝑥 uniformly at random, and forms a link to 𝑥 § (2) generate two random numbers 𝑦 and 𝑧 from geometric distributions with means 𝑞/(1 − 𝑞) and 𝑠𝑞/(1 − 𝑠𝑞) § (3) 𝑤 selects 𝑦 out-links and 𝑧 in-links of 𝑥 incident to nodes that were not yet visited and form out-links to them § (4) 𝑤 applies step (2) to the nodes found in step (3) w v Example: (1) Connect to a random node 𝑥 a (2) Sample 𝑦 =2, 𝑧 =1 c (3) Connect to 2 out- and 1 in-links of 𝑥 , namely 𝑏, 𝑐, 𝑑 b (4) Repeat the process for 𝑏, 𝑐, 𝑑 11/14/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 21

¡ Forest Fire generates graphs that densify and have shrinking diameter E(t) densification diameter diameter 1.32 N(t) N(t) 11/14/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 22

¡ Forest Fire also generates graphs with power-law degree distribution in-degree out-degree log count vs. log in-degree log count vs. log out-degree 11/14/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 23

¡ Fix backward probability r and vary forward Clique-like burning prob. p graph Increasing ¡ Notice a sharp diameter Constant transition diameter Sparse between sparse graph and clique-like Decreasing graphs diameter ¡ The “sweet spot” is very narrow 11/14/19 Jure Leskovec, Stanford CS224W: Machine Learning with Graphs, http://cs224w.stanford.edu 24