Efficient Bayesian inference for Copula Gaussian graphical models - PowerPoint PPT Presentation

Efficient Bayesian inference for Copula Gaussian graphical models A. Mohammadi, F. Abegaz and E. Wit University of Groningen, Netherlands a.mohammadi@rug.nl 29th International Workshop on Statistical Modelling Gottingen, Germmany July 17, 2013 A. Mohammadi, F. Abegaz and E. Wit (RUG) Copula Gaussian graphical modeling July 17, 2013 1 / 19

Motivation: Survey data What are sociological determinants and consequences of income? age inc child pdeg pinc deg pchild A. Mohammadi, F. Abegaz and E. Wit (RUG) Copula Gaussian graphical modeling July 17, 2013 2 / 19

Graphical models and Conditional Independence Gaussian graphical model: Graph G = ( V , E ) as N p (0 , Σ) | K = Σ − 1 is positive definite based on G M G = � � Relationship graph, conditional independence and Σ − 1 X i ⊥ X j | X V \{ i , j } ⇔ k ij = 0 , A. Mohammadi, F. Abegaz and E. Wit (RUG) Copula Gaussian graphical modeling July 17, 2013 3 / 19

Copula for ordinal data A. Mohammadi, F. Abegaz and E. Wit (RUG) Copula Gaussian graphical modeling July 17, 2013 4 / 19

Copula Gaussian grpahical models Model: Variables (X) and latent variables (Z) Z V ∼ N p (0 , K − 1 ) , V = { 1 , ..., p } , X v = F − 1 v ∈ V , (Φ( Z v )) , v Likelihood: extended rank likelihood (Hoff, 2007) p ( X | K , G , F ) = p ( D| K , G ) p ( X |D , K , G , F ) D = { Z ∈ R n × p : max { z k v : x ( k ) < x ( j ) v } < z ( j ) v : x ( k ) < x ( j ) < min { z k v }} v v v A. Mohammadi, F. Abegaz and E. Wit (RUG) Copula Gaussian graphical modeling July 17, 2013 5 / 19

Bayesian inference in Copula Gaussian graphial models Joint posterior distribution p ( K , G |D ) ∝ p ( D| K , G ) p ( K | G ) p ( G ) Prior for graph Discrete Uniform Truncated Poisson according to number of links Conjugate prior for precision matrix which satistifies G G-Wishart: W G ( b , D ) 1 � − 1 � I G ( b , D ) | K | ( b − 2) / 2 exp p ( K | G ) = 2 tr ( DK ) � � − 1 � | K | ( b − 2) / 2 exp I G ( b , D ) = 2 tr ( DK ) dK P G A. Mohammadi, F. Abegaz and E. Wit (RUG) Copula Gaussian graphical modeling July 17, 2013 6 / 19

Bayesian graph selection p ( G ) p ( data | G ) p ( G | data ) = � G ∈G p ( G ) p ( data | G ) Problems 1 Number of possible graphs = 2 ( p ( p − 1) / 2) 2 � p ( data | G ) = p ( data , θ G | G ) d θ G θ G Solutions: Trans-dimensional MCMC 1 Reversible-jump MCMC (Green 1995, 1999) 2 Birth-death MCMC (Mohammadi and Wit 2014) A. Mohammadi, F. Abegaz and E. Wit (RUG) Copula Gaussian graphical modeling July 17, 2013 7 / 19

Sampling algorithm: Birth-death MCMC BD-MCMC is continuous time Markov process Stationary distribution is joint posterior distribution of graph and parameters Compare with RJMCMC Both converge to joint posterior distribution of graph and parameters In BD-MCMC move between models are always accepted BD-MCMC converges much more faster compare with RJMCMC A. Mohammadi, F. Abegaz and E. Wit (RUG) Copula Gaussian graphical modeling July 17, 2013 8 / 19

BDMCMC algorithm Adding new link at birth and deleting link at death Theorem (Mohammadi and Wit, 2014) Our birth-death MCMC algorithm has stationary distribution P ( K , G | x ), if for each e = ( i , j ) δ e ( K ) P ( G , K \ ( k ij , k jj ) | x ) = β e ( K − e ) P ( G − e , K − e \ k jj | x ) . A. Mohammadi, F. Abegaz and E. Wit (RUG) Copula Gaussian graphical modeling July 17, 2013 9 / 19

Simple case of BD-MCMC A. Mohammadi, F. Abegaz and E. Wit (RUG) Copula Gaussian graphical modeling July 17, 2013 10 / 19

Simple case of BD-MCMC A. Mohammadi, F. Abegaz and E. Wit (RUG) Copula Gaussian graphical modeling July 17, 2013 11 / 19

Simple case of BD-MCMC A. Mohammadi, F. Abegaz and E. Wit (RUG) Copula Gaussian graphical modeling July 17, 2013 12 / 19

Simple case of BD-MCMC A. Mohammadi, F. Abegaz and E. Wit (RUG) Copula Gaussian graphical modeling July 17, 2013 13 / 19

Sampling scheme for CGGM Step 1 : Sample the latent data. we update the latent value z j from its full conditional distribution � Z j | Z V \{ j } = z V \{ j } ∼ N ( − K jr ′ z r ′ / K jj , 1 / K jj ) , r ′ truncated to the interval D j . Step 2 : (a). Calculate birth and death rates β e ( K ) = P ( G + e , K \ ( k ij , k jj ) | x ) for each e = ( i , j ) ∈ E , P ( G , K \ k jj | x ) δ e ( K ) = P ( G − e , K − e \ k jj | x ) for each e = ( i , j ) ∈ E . P ( G , K \ ( k ij , k jj ) | x ) Step 3 : Sampling new precision matrix: K + e or K − e A. Mohammadi, F. Abegaz and E. Wit (RUG) Copula Gaussian graphical modeling July 17, 2013 14 / 19

Statistical performence R-package BDgraph Package efficiently implements BDMCMC algorithm with C++ code linked to R A. Mohammadi, F. Abegaz and E. Wit (RUG) Copula Gaussian graphical modeling July 17, 2013 15 / 19

Result for Survey data age inc child pdeg pinc deg pchild A. Mohammadi, F. Abegaz and E. Wit (RUG) Copula Gaussian graphical modeling July 17, 2013 16 / 19

Simulation: Comparing with RJMCMC We consider a true graphical model with p = 6 as below N 6 (0 , Σ) | K = Σ − 1 ∈ P G � � M G = in which the precision matrix is  1 0 . 5 0 0 0 0 . 4  1 0 . 5 0 0 0     1 0 . 5 0 0   K =   1 0 . 5 0     1 0 . 5   1 A. Mohammadi, F. Abegaz and E. Wit (RUG) Copula Gaussian graphical modeling July 17, 2013 17 / 19

Some result Convergency plot: Cumulative occupancy fractions of all edges Convergency plot: Cumulative occupancy fractions of all edges 1.0 1.0 0.8 0.8 posterior link probability posterior link probability 0.6 0.6 0.4 0.4 0.2 0.2 0.0 0.0 0 10000 20000 30000 40000 50000 60000 0 10000 20000 30000 40000 50000 60000 iteration iteration Plot of cumulative occupancy fractions of all possible links for checking convergence of our BDMCMC (left) and RJCMCM (right). A. Mohammadi, F. Abegaz and E. Wit (RUG) Copula Gaussian graphical modeling July 17, 2013 18 / 19

References Mohammadi, A. and E. C. Wit (2014) Bayesian structure learning in Gaussian graphical models Bayesian Analysis , accepted Hoff, P.D. (2007) Extending the rank likelihood for semiparametric copula estimation The Annals of Applied Statistics , 1:265-283 Lenkoski, A. (2013) A direct sampler for G-Wishart variates Stat , 2:119-128 A. Mohammadi, F. Abegaz and E. Wit (RUG) Copula Gaussian graphical modeling July 17, 2013 19 / 19