Random Networks, Graphical Models and Exchangeability Alessandro - PowerPoint PPT Presentation

Exchangeability A finite deFinetti theorem Bidirected graphical models Random Networks, Graphical Models and Exchangeability Alessandro Rinaldo Carnegie Mellon University joint work with Steffen Lauritzen and Kayvan Sadeghi October 4, 2015 AMS Central Fall Sectional Meeting Loyola University Special Session on Algebraic Statistics and its Interactions with Combinatorics, Computation, and Network Science A. Rinaldo Random Networks, Exchangeability and Graphical Models 1/21

Exchangeability A finite deFinetti theorem Bidirected graphical models Outline Exchangeability of (infinite) networks. A finite deFinetti theorem and the dissociated property. Exchangeable and extendable finite networks are (mixtures of) bidirected graphical models. A. Rinaldo Random Networks, Exchangeability and Graphical Models 2/21

Exchangeability A finite deFinetti theorem Bidirected graphical models Statistical Network (Random Graph) Analysis Let L n be the set of simple labeled graphs on n nodes: ∣L n ∣ = 2 ( n 2 ) . A. Rinaldo Random Networks, Exchangeability and Graphical Models 3/21

Exchangeability A finite deFinetti theorem Bidirected graphical models Statistical Network (Random Graph) Analysis Let L n be the set of simple labeled graphs on n nodes: ∣L n ∣ = 2 ( n 2 ) . The nodes represent agents in some population of interest and the edges encode the relationships among them. A. Rinaldo Random Networks, Exchangeability and Graphical Models 3/21

Exchangeability A finite deFinetti theorem Bidirected graphical models Statistical Network (Random Graph) Analysis Let L n be the set of simple labeled graphs on n nodes: ∣L n ∣ = 2 ( n 2 ) . The nodes represent agents in some population of interest and the edges encode the relationships among them. Statistical Network Analysis Pose and estimate probability distributions on L n by modeling the joint occurrence of the ( n 2 ) random edges. A. Rinaldo Random Networks, Exchangeability and Graphical Models 3/21

Exchangeability A finite deFinetti theorem Bidirected graphical models Motivation: asymptotics of networks Let L = ⋃ n L , be the set of all finite (labeled, simple) graphs. A statistical model for L is a sequence { p n } n ∈ N of probability distributions, where p n is a probability distribution on L n . For n < m , let p n m denote the marginal of p m over L n . Consistency and Extendability A statistical model { p n } n ∈ N on L is consistent when, for any pair n < m , p n = p n m . (1) A probability distribution p n on L n is extendable when (1) holds ∀ m > n . A. Rinaldo Random Networks, Exchangeability and Graphical Models 4/21

Exchangeability A finite deFinetti theorem Bidirected graphical models Motivation: asymptotics of networks Let L = ⋃ n L , be the set of all finite (labeled, simple) graphs. A statistical model for L is a sequence { p n } n ∈ N of probability distributions, where p n is a probability distribution on L n . For n < m , let p n m denote the marginal of p m over L n . Consistency and Extendability A statistical model { p n } n ∈ N on L is consistent when, for any pair n < m , p n = p n m . (1) A probability distribution p n on L n is extendable when (1) holds ∀ m > n . Most network models are not consistent! A. Rinaldo Random Networks, Exchangeability and Graphical Models 4/21

Exchangeability A finite deFinetti theorem Bidirected graphical models Consistency via Exchangeability Let L ∞ be the set of (countably) infinite lableled, simple graphs. Every probability distrbution on L ∞ trivially specifies one consistent model! We impose one further restriction... A. Rinaldo Random Networks, Exchangeability and Graphical Models 5/21

Exchangeability A finite deFinetti theorem Bidirected graphical models Consistency via Exchangeability Let L ∞ be the set of (countably) infinite lableled, simple graphs. Every probability distrbution on L ∞ trivially specifies one consistent model! We impose one further restriction... Exchangeability A probability distribution on L ∞ is exchangeable when all finite isomorphic graphs have the same probabilities. A. Rinaldo Random Networks, Exchangeability and Graphical Models 5/21

Exchangeability A finite deFinetti theorem Bidirected graphical models Consistency via Exchangeability Let L ∞ be the set of (countably) infinite lableled, simple graphs. Every probability distrbution on L ∞ trivially specifies one consistent model! We impose one further restriction... Exchangeability A probability distribution on L ∞ is exchangeable when all finite isomorphic graphs have the same probabilities. Exchangeability is a most basic form of invariance, suitable to describe the "shape" of networks (large scale property). Labeled vs unlabeled. The exchangeability assumption is equivalent to define models on U n , the set of unlabaled graohs on n nodes, for all n . A. Rinaldo Random Networks, Exchangeability and Graphical Models 5/21

Exchangeability A finite deFinetti theorem Bidirected graphical models Exchangeability and graphons Analytic representation of exchangeable distributions The set of exchangeable distributions, with the topology of weak convergence, is a (Bauer) simplex. Denote its extreme points with E ∞ . p ∞ ∈ E ∞ if and only if, for every n and G ∈ L n ∞ ( G ) = ∫ [ 0 , 1 ] n ( 1 − f ( z i , z j )) dz 1 . . . z n , ∏ ∏ p n f ( z i , z j ) ( i , j )∈ E ( G ) ( i , j )/ ∈ E ( G ) where f ∶ [ 0 , 1 ] 2 → [ 0 , 1 ] is a (measurable) symmetric function, called a graphon. Graphons are unique up to measure preserving transformations of [ 0 , 1 ] . A. Rinaldo Random Networks, Exchangeability and Graphical Models 6/21

Exchangeability A finite deFinetti theorem Bidirected graphical models Exchangeability and graphons Analytic representation of exchangeable distributions The set of exchangeable distributions, with the topology of weak convergence, is a (Bauer) simplex. Denote its extreme points with E ∞ . p ∞ ∈ E ∞ if and only if, for every n and G ∈ L n ∞ ( G ) = ∫ [ 0 , 1 ] n f ( z i , z j ) ( 1 − f ( z i , z j )) dz 1 . . . z n , ∏ ∏ p n ( i , j )∈ E ( G ) ( i , j )/ ∈ E ( G ) where f ∶ [ 0 , 1 ] 2 → [ 0 , 1 ] is a (measurable) symmetric function, called a graphon. Graphons are unique up to measure preserving transformations of [ 0 , 1 ] . Vast literature: Aldous, Hoover, Kallenberg, Diaconis and Freedman, Chayes, Borgs and Lovász, etc ect... Key point: only the finite marginals of p ∞ ∈ E ∞ can be realized. General exchangeable models are mixtures of such distributions. A. Rinaldo Random Networks, Exchangeability and Graphical Models 6/21

Exchangeability A finite deFinetti theorem Bidirected graphical models Graphons and homomorphism densities For G ∈ L n and H ∈ L k with k ≤ n , the density homomorphism of H in G is t ( H , G ) = ∣ hom ( H , G )∣ . n k A. Rinaldo Random Networks, Exchangeability and Graphical Models 7/21

Exchangeability A finite deFinetti theorem Bidirected graphical models Graphons and homomorphism densities For G ∈ L n and H ∈ L k with k ≤ n , the density homomorphism of H in G is t ( H , G ) = ∣ hom ( H , G )∣ . n k Convergence of graph sequences = convergence of marginal probabilities A sequence { G n } n ∈ N converges if and only if, for some graphon f and each H ∈ L with k nodes, n →∞ t ( H , G n ) = ∫ [ 0 , 1 ] k ∏ f ( z i , z j ) dz 1 . . . z k = P ( H ⊆ G ′ ) , lim ( i , j )∈ E ( H ) ∞ , p ∞ ∈ E ∞ defined by f . G ′ a random graph distributed like the p k The sequence { t ( H , f )} H ∈ L of density homomorphisms uniquely specifies p ∞ . A. Rinaldo Random Networks, Exchangeability and Graphical Models 7/21

Exchangeability A finite deFinetti theorem Bidirected graphical models Finite Exchangeability But real networks are finite! So, what can be said about the set P n of exchangeable distribution on L n ? A. Rinaldo Random Networks, Exchangeability and Graphical Models 8/21

Exchangeability A finite deFinetti theorem Bidirected graphical models Finite Exchangeability But real networks are finite! So, what can be said about the set P n of exchangeable distribution on L n ? Finite exchangeability does not yield consistent models Finite exchangeable probability distributions marginalize to but need not be extendable to exchangeable distributions. A. Rinaldo Random Networks, Exchangeability and Graphical Models 8/21

Exchangeability A finite deFinetti theorem Bidirected graphical models Finite Exchangeability But real networks are finite! So, what can be said about the set P n of exchangeable distribution on L n ? Finite exchangeability does not yield consistent models Finite exchangeable probability distributions marginalize to but need not be extendable to exchangeable distributions. Our goal We would like to characterize the distributions in P n that are extendable. We seek to establish a parametric (finite dimensional) representation of all the distributions { p n ∞ , p ∞ ∈ E ∞ } . A. Rinaldo Random Networks, Exchangeability and Graphical Models 8/21