Limit Laws for Random Spatial Graphical Models Anima Anandkumar 1 - PowerPoint PPT Presentation

Limit Laws for Random Spatial Graphical Models Anima Anandkumar 1 Joseph Yukich 2 Alan Willsky 1 1 EECS, Massachusetts Institute of Technology, Cambridge, MA. USA 2 Dept. of Math., Lehigh University, Bethlehem, PA. USA IEEE ISIT 2010 Anandkumar et al. (MIT,Lehigh) Limit Laws ISIT ‘10 1 / 24

Introduction Motivating Examples Sensor network collecting data. Social network in a physical geographic Fusion center area. Locations of Nodes Irregular, far from a lattice Random node placement Observations at Nodes Correlated observations Dependent on locations of nodes Assumed to be a graphical model Anandkumar et al. (MIT,Lehigh) Limit Laws ISIT ‘10 2 / 24

Motivating Example: Ising Model Ising Model Used to model attractive forces between molecules Change of state related to phase transition Anandkumar et al. (MIT,Lehigh) Limit Laws ISIT ‘10 3 / 24

Motivating Example: Ising Model Ising Model Used to model attractive forces between molecules Change of state related to phase transition Anandkumar et al. (MIT,Lehigh) Limit Laws ISIT ‘10 3 / 24

Motivating Example: Ising Model Ising Model Used to model attractive forces between molecules Change of state related to phase transition Graphical Models under Random Node Placement How does irregular and random node placement affect properties compared to lattice structure? How do the limits of functions of observations depend on node placement and when do they exist? Anandkumar et al. (MIT,Lehigh) Limit Laws ISIT ‘10 3 / 24

Brief Problem Description Consider n randomly distributed nodes at locations V i ∈ V n making random observations Y V n . f ( x ) R = � n . . .... ... . . . . πλ V i Random Node Locations and Euclidean Graphs i.i.d. Points X i ∼ f ( x ) on unit ball B 1 Network scaled to a fixed density λ : V i = � n λ X i G ( V n ) is a graph on V n Anandkumar et al. (MIT,Lehigh) Limit Laws ISIT ‘10 4 / 24

Brief Problem Description Contd., Correlated Observations as a Graphical Model Y V n is a graphical model (Markov random field) with G ( V n ) as the graph. Functions of Node Observations Locally-defined functions involving sums of terms at individual nodes ◮ Example: mean value of the observations. n 1 � ξ (( V i , Y i ); ( V n , Y n )) , n → ∞ . n i =1 When do the limits exist? How do the limits depend on graph G and node placement distribution f ? Anandkumar et al. (MIT,Lehigh) Limit Laws ISIT ‘10 5 / 24

Summary of Results Limits of Functions of Random Spatial Graphical Models n 1 � lim n E [ ξ (( V i , Y i ); ( V n , Y n ))] = ξ ∞ . n →∞ i =1 Conditions for Existence of Limits ξ has bounded moments Weak dependence on data and position of nodes far away Related to degree-dependent percolation Relationship between Limiting Behavior and Graph Randomness Limiting constant ξ ∞ , as an explicit function of node placement distribution f and graph G Anandkumar et al. (MIT,Lehigh) Limit Laws ISIT ‘10 6 / 24

Related Work Classical Works on Gibbs Measures on Lattices Ising model is the most understood model Books by Georgii, Simon etc., Gibbs Measures on Trees Phase transition threshold based on maximum degree (Weitz) Recent Works on Gibbs Measures on Random Graphs Phase transitions in sparse random graphs (Dembo & Montanari, Mossel & Sly) Utilize locally tree-like property Limit Laws on Euclidean Random Graphs Graphs such as k-NNG, geometric random graph CLT, LLN for graph functionals (Penrose & Yukich) Do not cover correlated variables on random graphs Anandkumar et al. (MIT,Lehigh) Limit Laws ISIT ‘10 7 / 24

Outline Introduction 1 System Model 2 Limit Laws for Gibbs Functionals 3 Applications and Examples 4 Conclusion 5 Anandkumar et al. (MIT,Lehigh) Limit Laws ISIT ‘10 8 / 24

Dependency Graph and Graphical Model Consider an undirected graph G ( V ) , each vertex V i ∈ V is associated with a random variable Y i Y j Y i Y k Graphical Models also known as Markov Random Fields. Anandkumar et al. (MIT,Lehigh) Limit Laws ISIT ‘10 9 / 24

Dependency Graph and Graphical Model Consider an undirected graph G ( V ) , each vertex V i ∈ V is associated with a random variable Y i V \{N ( i ) ∪ i } N ( i ) i Y i ⊥ ⊥ Y V \{N ( i ) ∪ i } | Y N ( i ) Graphical Models also known as Markov Random Fields. Anandkumar et al. (MIT,Lehigh) Limit Laws ISIT ‘10 9 / 24

Dependency Graph and Graphical Model Consider an undirected graph G ( V ) , each vertex V i ∈ V is associated with a random variable Y i For any disjoint sets A, B, C such that C separates A and B , V \{N ( i ) ∪ i } B N ( i ) C i A Y A ⊥ ⊥ Y B | Y C Y i ⊥ ⊥ Y V \{N ( i ) ∪ i } | Y N ( i ) Graphical Models also known as Markov Random Fields. Anandkumar et al. (MIT,Lehigh) Limit Laws ISIT ‘10 9 / 24

Likelihood Function of Graphical Model Hammersley-Clifford Theorem’71 Let P be conditional pdf of observations with graph G ( v n ) , given locations V n = v n , P ( Y v n | V n = v n ) = 1 � Z exp[ Ψ c ( Y c )] . c ∈ C n where C n is the set of maximal cliques in G ( v n ) and Z is the normalization constant (partition function). Also, known as Gibbs distribution. Graphical Models with Pairwise Interactions P ( Y v n | V n = v n ) = 1 � Z exp[ Ψ i,j ( Y i , Y j )] . ( i,j ) ∈ G ( v n ) Example: Ising model has Ψ i,j ( Y i , Y j ) = βY i Y j where β is called inverse temperature. Anandkumar et al. (MIT,Lehigh) Limit Laws ISIT ‘10 10 / 24

Euclidean Random Graphs Random Node Placement f ( x ) R = � n . . i.i.d. .... ... . . Points X i ∼ f ( x ) on unit ball B 1 . . πλ Fixed density λ : V i = � n λ X i V i Stabilizing graph (Penrose-Yukich) Local graph structure not affected by far away points ( k -NNG, Disk) There is an a.s finite random radius R such that changes outside the ball do not affect the edges at origin. M. D. Penrose and J. E. Yukich, “Weak Laws Of Large Numbers In Geometric Probability,” Annals of Applied probability, vol. 13, no. 1, pp. 277-303, 2003 Anandkumar et al. (MIT,Lehigh) Limit Laws ISIT ‘10 11 / 24

Illustration of Stabilization: 1-NNG Anandkumar et al. (MIT,Lehigh) Limit Laws ISIT ‘10 12 / 24

Illustration of Stabilization: 1-NNG Anandkumar et al. (MIT,Lehigh) Limit Laws ISIT ‘10 12 / 24

Illustration of Stabilization: 1-NNG Anandkumar et al. (MIT,Lehigh) Limit Laws ISIT ‘10 12 / 24

Illustration of Stabilization: 1-NNG Anandkumar et al. (MIT,Lehigh) Limit Laws ISIT ‘10 12 / 24

Illustration of Stabilization: 1-NNG Anandkumar et al. (MIT,Lehigh) Limit Laws ISIT ‘10 12 / 24

Illustration of Stabilization: 1-NNG Anandkumar et al. (MIT,Lehigh) Limit Laws ISIT ‘10 12 / 24

Illustration of Stabilization: 1-NNG Anandkumar et al. (MIT,Lehigh) Limit Laws ISIT ‘10 12 / 24

Illustration of Stabilization: 1-NNG Anandkumar et al. (MIT,Lehigh) Limit Laws ISIT ‘10 12 / 24

Illustration of Stabilization: 1-NNG Anandkumar et al. (MIT,Lehigh) Limit Laws ISIT ‘10 12 / 24

Illustration of Stabilization: 1-NNG Anandkumar et al. (MIT,Lehigh) Limit Laws ISIT ‘10 12 / 24

Illustration of Stabilization: 1-NNG Anandkumar et al. (MIT,Lehigh) Limit Laws ISIT ‘10 12 / 24

Illustration of Stabilization: 1-NNG Anandkumar et al. (MIT,Lehigh) Limit Laws ISIT ‘10 12 / 24

Illustration of Stabilization: 1-NNG Anandkumar et al. (MIT,Lehigh) Limit Laws ISIT ‘10 12 / 24

Outline Introduction 1 System Model 2 Limit Laws for Gibbs Functionals 3 Applications and Examples 4 Conclusion 5 Anandkumar et al. (MIT,Lehigh) Limit Laws ISIT ‘10 13 / 24

Functions on Random Spatial Graphical Models Locally-defined functions involving sums of terms at individual nodes ξ (( V i , Y V i ); ( V n , Y V n )) = ξ ( Y V i ; Y N ( V i ) ) . ◮ Example: mean value of the observations. Result Under certain conditions n 1 � lim n E [ ξ (( V i , Y V i ); ( V n , Y V n ))] = ξ ∞ . n →∞ i =1 � ξ ∞ = E [ ξ (( V 0 , Y 0 ); ( P λf ( y ) , Y P λf ( y ) ))] f ( y ) dy, B 1 where P λ is Poisson process with intensity λ . For uniform node placement ( f ≡ 1) , ξ ∞ = E [ ξ (( V 0 , Y 0 ); ( P λ , Y P λ ))] . Anandkumar et al. (MIT,Lehigh) Limit Laws ISIT ‘10 14 / 24

Limiting Constant ξ ∞ via Poissonization n 1 � lim n E [ ξ (( V i , Y V i ); ( V n , Y V n ))] = ξ ∞ . n →∞ i =1 � ξ ∞ = E [ ξ (( V 0 , Y 0 ); ( P λf ( y ) , Y P λf ( y ) ))] f ( y ) dy. B 1 Origin n → ∞ Conditions for Existence of Limits ξ has bounded moments. Graph G is Stabilizing, i.e., finite radius of stabilization. P ( Y V n | V n ) is spatially mixing or in uniqueness regime. Anandkumar et al. (MIT,Lehigh) Limit Laws ISIT ‘10 15 / 24

Spatial Mixing or Uniqueness Regime Effect of changing value as fn. of distance Asymptotic independence between observation at a node and observations at nodes far away d TV ( P [ Y v | Y V = y V ] , P [ Y v | Y V = z V ]) ≤ δ ( dist G ( v, V )) , for any two feasible configurations y V , z V ∈ Y | V | , such that lim s →∞ δ ( s ) = 0 . Anandkumar et al. (MIT,Lehigh) Limit Laws ISIT ‘10 16 / 24

Outline Introduction 1 System Model 2 Limit Laws for Gibbs Functionals 3 Applications and Examples 4 Conclusion 5 Anandkumar et al. (MIT,Lehigh) Limit Laws ISIT ‘10 17 / 24