Network determination based on birth-death MCMC inference A. - PowerPoint PPT Presentation

N ETWORK B IRTH - DEATH MCMC METHOD S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES Network determination based on birth-death MCMC inference A. Mohammadi and E. Wit February 4, 2013

N ETWORK B IRTH - DEATH MCMC METHOD S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES W HAT IS TALK ABOUT ? Problem ◮ High-dimensional cases: p ( p − 1 ) / 2 ≫ n ◮ Bayesian approches : Not fast ◮ glasso : Sensitivity to tuning parameters Solution ◮ We proposed Bayesian method which is fast and accurate ◮ Implement to R package: BDgraph Trans-dimensional MCMC ◮ Reversible-jump MCMC ◮ Birth-Death MCMC

N ETWORK B IRTH - DEATH MCMC METHOD S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES Network Gaussian graphical model with respect to graph G = ( V , E ) as N p ( 0 , Σ) | K = Σ − 1 is positive definite based on G � � M G = Pairwise Markov property X i ⊥ X j | X V \{ i , j } ⇔ k ij = 0 ,

N ETWORK B IRTH - DEATH MCMC METHOD S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES B IRTH - DEATH PROCESS ◮ Spacial birth-death process: Preston (1976) ◮ Brith-death MCMC: Stephen (2000) in mixture models

N ETWORK B IRTH - DEATH MCMC METHOD S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES B IRTH - DEATH MCMC DESIGN General birth-death process ◮ Continuous Markov process ◮ Birth and death events are independent Poisson processes ◮ Time of birth or death event is exponentially distributed Birth-death process in GGM ◮ Adding new edge in birth time and deleting edge in death time

N ETWORK B IRTH - DEATH MCMC METHOD S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES S IMPLE CASE

N ETWORK B IRTH - DEATH MCMC METHOD S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES S IMPLE CASE

N ETWORK B IRTH - DEATH MCMC METHOD S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES S IMPLE CASE

N ETWORK B IRTH - DEATH MCMC METHOD S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES S IMPLE CASE

N ETWORK B IRTH - DEATH MCMC METHOD S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES S IMPLE CASE

N ETWORK B IRTH - DEATH MCMC METHOD S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES S IMPLE CASE

N ETWORK B IRTH - DEATH MCMC METHOD S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES B ALANCE CONDITION Preston (1976): Backward Kolmogorov If balance conditions are hold, process converges to unique stationary distribution.

N ETWORK B IRTH - DEATH MCMC METHOD S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES P ROPOSED BIRTH - DEATH MCMC ALGORITHM Proposal birth and death rates β ξ ( K ) = λ b , new link ξ = ( i , j ) b ξ ( k ξ ) p ( G − ξ , K − ξ | x ) δ ξ ( K ) = λ b , existing link ξ = ( i , j ) p ( G , K | x ) Proposal birth-death MCMC algorithm Starting with initial graph: Step 1 : (a). Calculate birth and death rates (b). Calculate waiting time, λ ( K ) = 1 / ( β ( K ) + δ ( K )) (c). Simulate type of jump, birth or death Step 2: Sampling from new precision matrix: K + ξ or K − ξ

N ETWORK B IRTH - DEATH MCMC METHOD S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES S AMPLING BIRTH - DEATH MCMC ALGORITHM

N ETWORK B IRTH - DEATH MCMC METHOD S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES ◮ Birth-death MCMC algorithm for general case ...

N ETWORK B IRTH - DEATH MCMC METHOD S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES S PECIFIC ELEMENT OF BDMCMC METHOD ◮ Prior distributions ◮ Computing death rates ◮ Sampling from precision matrix

N ETWORK B IRTH - DEATH MCMC METHOD S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES P ROPOSED PRIOR DISTRIBUTIONS Prior for graph ◮ Discrete Uniform ◮ Truncated Poisson according to number of links Prior for precision matrix ◮ G-Wishart: W G ( b , D ) � − 1 � p ( K | G ) ∝ | K | ( b − 2 ) / 2 exp 2 tr ( DK ) − 1 � � � | K | ( b − 2 ) / 2 exp I G ( b , D ) = 2 tr ( DK ) dK P G

N ETWORK B IRTH - DEATH MCMC METHOD S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES G-W ISHART DISTRIBUTION Sampling from G-Wishart distribution ◮ Accept-reject algorithm ◮ Metropolis-Hastings algorithm ◮ Block Gibbs sampler ◮ According to maximum cliques ◮ Edgewise block Gibbs sampler

N ETWORK B IRTH - DEATH MCMC METHOD S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES C OMPUTING DEATH RATES p ( G − ξ , K − ξ | x ) δ ξ ( K ) = p ( G , K | x ) γ b b ξ ( k ξ ) � ( b ∗ − 2 ) / 2 � | K − ξ | I G ( b , D ) − 1 � � 2tr ( D ∗ ( K − ξ − K )) = exp γ b b ξ ( | K | I G − ξ ( b , D ) Limitation Algorithm is very slow !!

N ETWORK B IRTH - DEATH MCMC METHOD S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES Ratio of normalizing constant � b + ν i � Γ E G [ f T ( ψ ν )] I G − ξ ( b , D ) = 2 √ π t ii t jj I G ( b , D ) 2 � b + ν i − 1 � E G − ξ [ f T ( ψ ν )] Γ 2 Plot for ratio of normalizing constants 1.20 1.15 ratio of expectation 1.10 1.05 1.00 0 50 100 150 200 250 300 number of nodes

N ETWORK B IRTH - DEATH MCMC METHOD S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES Death rates for high-dimensional cases � b + ν i � � ( b ∗ − 2 ) / 2 Γ � | K − ξ | 2 √ π t ii t jj 2 δ ξ ( K ) = � b + ν i − 1 � | K | Γ 2 � − 1 � 2tr ( D ∗ ( K − ξ − K )) × exp γ b b ξ ( k ξ ) R package We compile our method into BDgraph package which is available from CRAN web site

N ETWORK B IRTH - DEATH MCMC METHOD S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES S IMULATION : 8 NODES N 8 ( 0 , Σ) | K = Σ − 1 ∈ P G � � M G =   1 . 5 0 0 0 0 0 . 4 1 . 5 0 0 0 0 0     1 . 5 0 0 0 0      1 . 5 0 0 0    K =   . 5 1 0 0     1 . 5 0     . 5  1    1

N ETWORK B IRTH - DEATH MCMC METHOD S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES S OME RESULT Effect of Sample size Number of data 20 30 40 60 80 100 150 p(true graph | data) 0.018 0.067 0.121 0.2 0.22 0.35 0.43 false discovery 1 0 0 0 0 0 0 false negative 0 0 0 0 0 0 0

N ETWORK B IRTH - DEATH MCMC METHOD S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES S OME RESULT   1 1 0 . 03 0 . 06 0 . 02 0 . 02 0 . 03 1 0 . 04 0 . 03 0 . 02 0 . 03 0 . 03 1 1     1 1 0 . 06 0 . 04 0 . 06 0 . 03       1 1 0 . 05 0 . 04 0 . 03   ˆ p ξ = .   1 1 0 . 05 0 . 13     1 1 0 . 13     1 1     1   1 . 3 0 . 6 0 0 0 0 0 0 . 5 1 . 4 0 . 5 0 0 0 0 0     1 0 . 5 0 0 0 0      1 . 2 0 . 6  0 0 0 ˆ   K = .   1 . 3 0 . 4 0 0     0 . 9 0 . 5 0     0 . 9 0 . 4     1

N ETWORK B IRTH - DEATH MCMC METHOD S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES S IMULATION : 120 NODES N 120 ( 0 , Σ) | K = Σ − 1 ∈ P G � � M G = , ◮ n = 1000 ≪ 7260 ◮ Priors: K ∼ W G ( 3 , I 120 ) and G ∼ TU ( all possible graphs ) ◮ 10000 iterations and 5000 iterations as burn-in Result ◮ Time 190 minutes ◮ p(true graph | data) = 0 . 41 which is most probable graph

N ETWORK B IRTH - DEATH MCMC METHOD S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES C ELL SIGNALING DATA Flow cytometry data with 11 proteins from Sachs et al. (2005) (Left) Result from our algorithm (Right) Result from Sachs et al (2005) Friedman et al (2008): full graph according to g-lasso

N ETWORK B IRTH - DEATH MCMC METHOD S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES C ONCLUSION

N ETWORK B IRTH - DEATH MCMC METHOD S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES Thanks for your attention References M OHAMMADI , A. AND E. C. W IT (2012) Efficient birth-death MCMC inference for Gaussian graphical models. arXiv preprint arXiv:1210.5371 W ANG , H. AND S. L I (2012) Efficient Gaussian graphical model determination under G-Wishart prior distributions. Electronic Journal of Statistics, 6:168-198 A TAY -K AYIS , A. AND H. M ASSAM (2005) A Monte Carlo method for computing the marginal likelihood in nondecomposable Gaussian graphical models. Biometrika Trust, 92(2):317-335 PRESTON , C. J. (1976) Special birth-and-death processes. Bull. Inst. Internat. Statist., 34:1436-1462