Grbner Bases of Gaussian Graphical Models Alex Fink, Jenna Rajchgot, - PowerPoint PPT Presentation

Gröbner Bases of Gaussian Graphical Models Alex Fink, Jenna Rajchgot, Seth Sullivant Queen Mary University, University of Michigan, North Carolina State University October 4, 2015 Seth Sullivant (NCSU) Gaussian Graphical Models October 4, 2015 1 / 14

Gaussian Graphical Models Graphical models are a flexible framework for building statistical models on large collections of random variables. Edges of different types represent different types of interactions between neighboring random variables. directed edges: i → j bidirected edges: i ↔ j undirected edges: i − j Graph is used to express both conditional independence structures parametric representations of the model. For jointly normal random variables, graph structure relates variables to their neighbors via linear relationships with possibly correlated error terms. Seth Sullivant (NCSU) Gaussian Graphical Models October 4, 2015 2 / 14

Mixed Graphs G = ( V , B , D ) graph with directed edges D ( i → j ) and bidirected edges B ( i ↔ j ) Vertex set V = [ m ] := { 1 , 2 , . . . m } G is acyclic: i → j ∈ D implies i < j . 4 2 3 1 Seth Sullivant (NCSU) Gaussian Graphical Models October 4, 2015 3 / 14

Gaussian Graphical Models Gaussian graphical model is a statistical model that associates a family of normal distributions to the graph G = ( V , B , D ) . PD m = cone of symmetric positive definite matrices Let PD ( B ) := { M ∈ PD m : M ij = 0 if i � = j and i ↔ j / ∈ B } ǫ ∈ R m , ǫ ∼ N ( 0 , Ω) with Ω ∈ PD ( B ) For i → j ∈ D , let λ ij ∈ R Define X ∈ R m recursively by � X j = λ ij X i + ǫ j . i : i → j ∈ D Seth Sullivant (NCSU) Gaussian Graphical Models October 4, 2015 4 / 14

Example 4 3 1 2  ω 11 0 0 0  0 ω 22 0 ω 24   ǫ ∼ N ( 0 , Ω) Ω =   ω 33 0 0 0   0 ω 42 0 ω 44 X 1 = ǫ 1 , X 2 = λ 12 X 1 + ǫ 2 , X 3 = λ 13 X 1 + λ 23 X 2 + ǫ 3 , X 4 = λ 34 X 3 + ǫ 4 Seth Sullivant (NCSU) Gaussian Graphical Models October 4, 2015 5 / 14

Matrix Factorization Let Λ m × m upper triangular matrix such that � λ ij if i → j ∈ D Λ ij = 0 otherwise. Proposition X from the graph G = ( V , B , D ) is distributed N ( 0 , Σ) where Σ = ( I − Λ) − T Ω( I − Λ) − 1 . Note that ( I − Λ) − 1 = I + Λ + Λ 2 + · · · + Λ m − 1 . Seth Sullivant (NCSU) Gaussian Graphical Models October 4, 2015 6 / 14

The Algebraic Perspective Let R D = { Λ ∈ R m × m : λ ij = 0 if i → j / ∈ D } PD ( B ) := { M ∈ PD m : M ij = 0 if i � = j and i ↔ j / ∈ B } Definition The Gaussian graphical model M G ⊆ PD m consists of all covariance matrices Σ , that arise for some choice of Λ ∈ R D and Ω ∈ PD ( B ) . More algebraically, we have a polynomial map φ G : R D × PD ( B ) → PD m , φ G (Λ , Ω) = ( I − Λ) − T Ω( I − Λ) − 1 . M G = im φ G . We would like to “understand” φ G and M G . Seth Sullivant (NCSU) Gaussian Graphical Models October 4, 2015 7 / 14

Constraints on Gaussian Graphical Models Problem Find a generating set or Gröbner basis of I G = I ( M G ) = { f ∈ R [ σ ij : i , j ∈ [ m ]] : f (Σ) = 0 for all Σ ∈ M G } . 1 2 I G = �| Σ 12 , 13 | , | Σ 123 , 234 |� 4 3 Seth Sullivant (NCSU) Gaussian Graphical Models October 4, 2015 8 / 14

Constraints on Gaussian Graphical Models Problem Find a generating set or Gröbner basis of I G = I ( M G ) = { f ∈ R [ σ ij : i , j ∈ [ m ]] : f (Σ) = 0 for all Σ ∈ M G } . 1 2 I G = �| Σ 12 , 13 | , | Σ 123 , 234 |� 4 3 4 2 3 1 � σ 12 σ 13 σ 14 σ 23 − σ 11 σ 14 σ 2 23 − σ 12 σ 2 I G = 13 σ 24 + σ 11 σ 13 σ 23 σ 24 − σ 2 12 σ 14 σ 33 + σ 11 σ 14 σ 22 σ 33 + σ 2 12 σ 13 σ 34 − σ 11 σ 13 σ 22 σ 34 � Seth Sullivant (NCSU) Gaussian Graphical Models October 4, 2015 8 / 14

Trek Separation A trek from i to j is a path in G from i to j with no sequence of edges k → l ← m , k ↔ l ← m , k → l ↔ m , or k ↔ l ↔ m . Definition Let A , B , C , and D be four subsets of V ( G ) (not necessarily disjoint). We say that ( C , D ) t-separates A from B if every trek from A to B passes through either a vertex in C on the A -side of the trek, or a vertex in D on the B -side of the trek. C D A B Seth Sullivant (NCSU) Gaussian Graphical Models October 4, 2015 9 / 14

Theorem (S-Talaska-Draisma) The matrix Σ A , B has rank ≤ d if and only if there are C , D ⊂ [ n ] with # C + # D ≤ d such that ( C , D ) t-separate A from B in G. Then all ( d + 1 ) × ( d + 1 ) minors of Σ A , B belong to I G . Example c B A ( { c } , { c } ) t -separates A from B . Σ A , B has rank at most 2. All 3 × 3 minors of Σ A , B belong to I G . Seth Sullivant (NCSU) Gaussian Graphical Models October 4, 2015 10 / 14

When do Determinantal Constraints Generate I G ? Question What conditions on the graph G guarantee that the t -separation determinantal constraints generate I G ? Theorem (Sullivant 2008) 2 1 If G is a tree, then I G is generated in degree 1 and 2 by conditional 3 independence constraints. In particular, I G generated by the t-separation 5 4 determinantal constraints. Proof Sketch. For trees I G is a toric ideal. Do binomial manipulations. Seth Sullivant (NCSU) Gaussian Graphical Models October 4, 2015 11 / 14

4 5 6 3 1 2 Definition A generalized Markov chain is a mixed graph G = ( V , B , D ) such that: If i → j ∈ D then i < j , If i → j ∈ D and i ≤ k < l ≤ j then k → l ∈ D , and If i ↔ j ∈ B and i ≤ k < l ≤ j then k ↔ l ∈ B . Theorem (Fink-Rajchgot-S (2015)) If G is a generalized Markov chain then I G is generated by the t-separation determinantal constraints implied by G, and they form a Gröbner basis in a suitable lexicographic term order. Proof Sketch. Relate generalized Markov chain parametrization to parametrization of type B matrix Schubert varieties. Seth Sullivant (NCSU) Gaussian Graphical Models October 4, 2015 12 / 14

Further Results and Open Problems We have extended t -separation characterization of determinantal constraints to ancestral graphs and AMP chain graphs. Connections to toric varieties and matrix Schubert varieties allow us to characterize the vanishing ideals for some graphs. How to determine general combinatorial descriptions of other hidden variable constraints? � � σ 11 σ 11 σ 12 σ 13 � � � � σ 12 σ 12 σ 22 σ 23 3 4 1 2 � � = 0 � � 0 σ 13 σ 23 σ 33 � � � � 0 σ 14 σ 24 σ 34 � � Seth Sullivant (NCSU) Gaussian Graphical Models October 4, 2015 13 / 14

References A. Fink, J. Rajchgot, S. Sullivant. Symmetric matrix Schubert varieties and Gaussian graphical models. (2015) In preparation. Knutson, Allen. Frobenius splitting, point-counting, and degeneration. (2009) :0911.4941 Knutson, Allen; Miller, Ezra. Gröbner geometry of Schubert polynomials. Ann. of Math. (2) 161 (2005), no. 3, 1245–1318. S. Sullivant. Algebraic geometry of Gaussian Bayesian networks. Adv. in Appl. Math. 40 (2008), no. 4, 482–513. 0704.0918 S. Sullivant, K. Talaska and J. Draisma. Trek separation for Gaussian graphical models. Annals of Statistics 38 no.3 (2010) 1665-1685 0812.1938 Seth Sullivant (NCSU) Gaussian Graphical Models October 4, 2015 14 / 14