Robust Estimation of Tree Structured Gaussian Graphical Models Ashish Katiyar, Jessica Hoffmann, Constantine Caramanis The University of Texas at Austin
Undirected Probabilistic Graphical Models • Represent conditional independence relations among random variables in the form of X₁ X 2 a graph. X 7 X 3 • Any random variable X 6 conditioned on the random variable it has an edge with, is X 5 X 4 independent of all the remaining random variables.
Why Graphical Models? • Identify interactions among Sample Data variables in large systems. (e.g. Gene Interaction Networks.) Graphical Model • Makes inference in large scale systems tractable. Inference
Gaussian Graphical Models X X 0 0 0 0 0 • X X X 0 0 0 0 - Jointly 0 X X X 0 X 0 = Gaussian random variables with 0 0 X X X 0 0 0 0 0 X X 0 0 covariance . 0 0 X 0 0 X X 0 0 0 0 0 X X • Support of inverse covariance X₁ X 2 gives the graphical model structure. X 7 X 3 X 6 X 5 X 4
Effect Of Noise • Additive Gaussian noise in the random variables breaks down X₁ X 2 the conditional independence. X 7 X 3 • Intuitively – Noisy samples do not convey the whole X 6 information. X 5 X 4
Effect Of Noise • Additive Gaussian noise in the random variables breaks down X₁ X 2 the conditional independence. X 7 X 3 • Intuitively – Noisy samples do not convey the whole X 6 information. X 5 X 4 - Noisy node X 3 - Extra Edges
Effect Of Noise • Additive Gaussian noise in the random variables breaks down X₁ X 2 the conditional independence. X 7 X 3 • Example: if X – Y – Z is a Markov chain, then X and Z are no longer X 6 independent when conditioned on a noisy version of Y. X 5 X 4
Problem Statement • Suppose the graphical model have a tree structure . = + X₁ X 2 X₁ X 2 • We observe . X 7 X 3 X 7 where is an unknown X 3 positive diagonal noise matrix. X 6 X 6 X 4 X 4 X 5 X 5 • Goal: recover given .
Bad News! Unidentifiability • Even for arbitrarily small noise True Tree 1 2 3 the problem is unidentifiable! Noisy graph 1 2 3 • There are covariance matrices that differ only on diagonal (a) Underlying tree and the noisy graphical model. entries, but their inverses have 1 2 3 different sparsity pattern. 1 1 1 3 2 2 2 2 1 3 3 3 (b) Graphical models for 3 different decompositions of .
Good News! Limited Unidentifiability • The ambiguity in tree structure 8 is highly limited. 7 1 6 9 • The only ambiguity is between a 3 4 5 leaf node and its immediate neighbor. 10 11 12 13 2 14 Trees formed by permuting nodes within the dotted regions form an equivalence class .
Proof - Key Idea 0 14 • Off-diagonal covariance entries have information about the tree 1 2 structure. 3 4 5 6 • They can be used to categorize 7 8 10 11 12 13 9 any set of 4 nodes as a star or non-star . Node Set Classification {0, 1, 3, 7} Non-Star • Non-star – Exactly 2 nodes lie in {7, 8, 9, 10} Non-Star one subtree. {14, 2, 10, 11} Non-Star {7, 9, 1, 6} Star {1, 14, 10, 6} Star
Proof – Key Idea • Categorization of any set of 4 0 nodes as star/non-star defines 14 all possible partitions in 2 subtrees with minimum 2 nodes. 1 2 3 4 5 6 • Thus the off diagonal elements define the tree upto the 7 8 9 10 11 12 13 equivalence class .
Identifiability with Side Information • Diagonal Majorization Condition – If the precision matrix is known to have diagonal entries greater than the absolute off diagonal entries. • Minimum Eigenvalue Condition – If a lower bound on the minimum eigenvalue of the covariance is known and the noise variance is smaller than this lower bound.
Cluster Tree EC – Equivalence Cluster EC 0 0 14 EC 2 EC 0 1 2 EC 1 EC 2 EC 1 3 5 6 4 7 8 10 11 12 13 9 EC 6 EC 3 EC 4 EC 5 EC 3 EC 4 EC 5 EC 6 Cluster Tree Original Tree
Algorithm - Initialization • Split the tree in 2 subtrees. Root Equivalence Clusters 0 • Find the root equivalence 14 equivalence cluster of each 1 2 subtree. 4 3 5 6 7 8 10 11 12 13 9
Algorithm – Recursion step
Conclusion • Unidentifiability of learning tree structured Gaussian Graphical Models in presence of noise. • Identifiability conditions with side information. • Algorithm to find the tree structure.
Recommend
More recommend
