Implementing discrete approximations to continuous mixture - PowerPoint PPT Presentation

Implementing discrete approximations to continuous mixture distributions Christian R¨ over Department of Medical Statistics University Medical Center G¨ ottingen December 5, 2014 C. R¨ over Implementing mixture approximations December 5, 2014 1 / 31

Overview mixture distributions meta analysis example discrete ‘grid’ approximations design strategy / algorithm example application C. R¨ over Implementing mixture approximations December 5, 2014 2 / 31

Mixture distributions mixture distribution: a convex combination of “component” distributions “a distribution whose parameters are random variables” (“conditional”) distribution with density p ( y | x ) “parameter” x follows a distribution p ( x ) � marginal distribution of y is p ( y ) = X p ( y | x ) d p ( x ) x discrete: p ( y ) = � i p ( y | x i ) p ( x i ) ubiquitous in many applications Student- t distribution negative binomial distribution marginal distributions . . . C. R¨ over Implementing mixture approximations December 5, 2014 3 / 31

Meta analysis Context: random-effects meta-analysis # 1 # 2 # 3 # 4 have: estimates y i # 5 standard errors σ i # 6 # 7 want: combined estimate ˆ Θ 120 140 160 180 200 220 240 effect Θ C. R¨ over Implementing mixture approximations December 5, 2014 4 / 31

Meta analysis Context: random-effects meta-analysis # 1 # 2 # 3 # 4 have: estimates y i # 5 standard errors σ i # 6 # 7 want: combined estimate ˆ Θ 120 140 160 180 200 220 240 effect Θ C. R¨ over Implementing mixture approximations December 5, 2014 4 / 31

Meta analysis Context: random-effects meta-analysis # 1 # 2 # 3 # 4 have: estimates y i # 5 standard errors σ i # 6 # 7 want: combined estimate ˆ Θ Θ 120 140 160 180 200 220 240 effect Θ C. R¨ over Implementing mixture approximations December 5, 2014 4 / 31

Meta analysis Context: random-effects meta-analysis # 1 # 2 # 3 # 4 have: estimates y i # 5 standard errors σ i # 6 # 7 want: combined estimate ˆ Θ Θ 120 140 160 180 200 220 240 effect Θ C. R¨ over Implementing mixture approximations December 5, 2014 4 / 31

Meta analysis The random effects model assume: 2 + τ 2 ) y i ∼ Normal (Θ , σ i C. R¨ over Implementing mixture approximations December 5, 2014 5 / 31

Meta analysis The random effects model assume: 2 + τ 2 ) y i ∼ Normal (Θ , σ i ingredients: Data : Parameters : estimates y i true parameter value Θ standard errors σ i heterogeneity τ C. R¨ over Implementing mixture approximations December 5, 2014 5 / 31

Meta analysis The random effects model assume: 2 + τ 2 ) y i ∼ Normal (Θ , σ i ingredients: Data : Parameters : estimates y i true parameter value Θ standard errors σ i heterogeneity τ C. R¨ over Implementing mixture approximations December 5, 2014 5 / 31

Meta analysis The random effects model assume: 2 + τ 2 ) y i ∼ Normal (Θ , σ i ingredients: Data : Parameters : estimates y i true parameter value Θ standard errors σ i heterogeneity τ C. R¨ over Implementing mixture approximations December 5, 2014 5 / 31

Meta analysis The random effects model assume: 2 + τ 2 ) y i ∼ Normal (Θ , σ i ingredients: Data : Parameters : estimates y i true parameter value Θ standard errors σ i heterogeneity τ C. R¨ over Implementing mixture approximations December 5, 2014 5 / 31

Meta analysis The random effects model assume: 2 + τ 2 ) y i ∼ Normal (Θ , σ i ingredients: Data : Parameters : estimates y i true parameter value Θ standard errors σ i heterogeneity τ Θ ∈ R of primary interest τ ∈ R + nuisance parameter: account for (potential) incompatibility C. R¨ over Implementing mixture approximations December 5, 2014 5 / 31

Meta analysis example Motivation: background 0.06 # 1 0.05 # 2 marginal posterior p ( Θ ) # 3 0.04 # 4 0.03 # 5 # 6 0.02 # 7 0.01 Θ 0.00 120 140 160 180 200 220 240 140 160 180 200 effect Θ effect Θ estimation: via marginal posterior distribution of parameter Θ C. R¨ over Implementing mixture approximations December 5, 2014 6 / 31

Meta analysis example Motivation: two-parameter model & marginals 0.06 190 0.05 0.05 180 marginal posterior p ( Θ ) marginal posterior p ( τ ) 95% 0.04 0.04 170 effect Θ 0.03 0.03 160 0.02 0.02 150 50% 90% 0.01 0.01 140 99% 0.00 0.00 130 0 10 20 30 40 0 20 40 60 80 140 160 180 200 heterogeneity τ heterogeneity τ effect Θ two unknowns: joint & marginal posterior distributions C. R¨ over Implementing mixture approximations December 5, 2014 7 / 31

Meta analysis example Motivation: two-parameter model, conditionals & marginals here: easy to derive one of the marginal s: p ( τ | y ) and conditional posteriors p (Θ | τ, y ) p ( τ | y ) = . . . (. . . function of y i , σ i ,. . . ) p (Θ | τ, y ) = Normal ( µ = f 1 ( τ ) , σ = f 2 ( τ )) but main interest in other marginal: p (Θ | y ) � p (Θ | y ) = p (Θ | τ, y ) p ( τ | y ) d τ is a mixture distribution � �� � � �� � conditional marginal C. R¨ over Implementing mixture approximations December 5, 2014 8 / 31

Meta analysis example Motivation: two-parameter model, conditionals & marginals 190 0.05 180 marginal posterior p ( τ ) 0.04 170 effect Θ 0.03 160 0.02 150 0.01 140 130 0.00 0 10 20 30 40 0 10 20 30 40 50 heterogeneity τ heterogeneity τ 0.10 0.06 conditional posterior p ( Θ | τ i ) 0.08 0.05 marginal posterior p ( Θ |y ) 0.04 0.06 0.03 0.04 0.02 0.02 0.01 0.00 0.00 130 140 150 160 170 180 190 130 140 150 160 170 180 190 effect Θ effect Θ C. R¨ over Implementing mixture approximations December 5, 2014 9 / 31

Meta analysis example Motivation: two-parameter model, conditionals & marginals 190 0.05 conditional mean + sd 180 marginal posterior p ( τ ) 0.04 170 effect Θ 0.03 conditional mean 160 0.02 150 0.01 140 conditional mean − sd 130 0.00 0 10 20 30 40 0 10 20 30 40 50 heterogeneity τ heterogeneity τ 0.10 0.06 conditional posterior p ( Θ | τ i ) 0.08 0.05 marginal posterior p ( Θ |y ) 0.04 0.06 0.03 0.04 0.02 0.02 0.01 0.00 0.00 130 140 150 160 170 180 190 130 140 150 160 170 180 190 effect Θ effect Θ C. R¨ over Implementing mixture approximations December 5, 2014 9 / 31

Meta analysis example Motivation: two -parameter model, conditionals & marginals τ 1 τ 1 190 0.05 180 marginal posterior p ( τ ) 0.04 170 effect Θ 0.03 160 0.02 150 0.01 140 130 0.00 0 10 20 30 40 0 10 20 30 40 50 heterogeneity τ heterogeneity τ 0.10 τ = τ 1 0.06 conditional posterior p ( Θ | τ i ) 0.08 0.05 marginal posterior p ( Θ |y ) 0.04 0.06 0.03 0.04 0.02 0.02 0.01 0.00 0.00 130 140 150 160 170 180 190 130 140 150 160 170 180 190 effect Θ effect Θ C. R¨ over Implementing mixture approximations December 5, 2014 9 / 31

Meta analysis example Motivation: two -parameter model, conditionals & marginals τ 1 τ 2 τ 3 τ 4 τ 1 τ 2 τ 3 τ 4 190 0.05 180 marginal posterior p ( τ ) 0.04 170 effect Θ 0.03 160 0.02 150 0.01 140 130 0.00 0 10 20 30 40 0 10 20 30 40 50 heterogeneity τ heterogeneity τ 0.10 τ = τ 1 0.06 conditional posterior p ( Θ | τ i ) 0.08 0.05 marginal posterior p ( Θ |y ) 0.04 0.06 τ = τ 2 0.03 0.04 τ = τ 3 0.02 τ = τ 4 0.02 0.01 0.00 0.00 130 140 150 160 170 180 190 130 140 150 160 170 180 190 effect Θ effect Θ C. R¨ over Implementing mixture approximations December 5, 2014 9 / 31

Meta analysis example Motivation: two -parameter model, conditionals & marginals τ 1 τ 2 τ 3 τ 4 τ 1 τ 2 τ 3 τ 4 190 0.05 180 marginal posterior p ( τ ) 0.04 170 effect Θ 0.03 160 0.02 150 0.01 140 130 0.00 0 10 20 30 40 0 10 20 30 40 50 heterogeneity τ heterogeneity τ 0.10 τ = τ 1 0.06 conditional posterior p ( Θ | τ i ) 0.08 0.05 marginal posterior p ( Θ |y ) 0.04 0.06 τ = τ 2 0.03 0.04 τ = τ 3 0.02 τ = τ 4 0.02 0.01 0.00 0.00 130 140 150 160 170 180 190 130 140 150 160 170 180 190 effect Θ effect Θ C. R¨ over Implementing mixture approximations December 5, 2014 9 / 31

Meta analysis example Questions approximating the continuous mixture through a discrete set of points in τ . . . actual marginal: � p (Θ) = p (Θ | τ ) p ( τ ) d τ approximation: � p (Θ) ≈ p (Θ | τ i ) π i i Questions: how to set up the discrete grid of points? how well can we approximate? do we have a handle on accuracy? C. R¨ over Implementing mixture approximations December 5, 2014 10 / 31

Meta analysis example Motivation: discretizing a mixture 0.10 τ 1 τ 2 τ 3 τ 4 τ = τ 1 190 conditional posterior p ( Θ | τ i ) 180 0.08 170 0.06 effect Θ τ = τ 2 160 0.04 τ = τ 3 150 τ = τ 4 0.02 140 0.00 130 0 10 20 30 40 130 140 150 160 170 180 190 heterogeneity τ effect Θ Note: conditional distributions p (Θ | τ, y ) are very different for τ 1 and τ 2 and rather similar for τ 3 and τ 4 . idea: may need fewer bins for larger τ values...? . . . bin spacing based on similarity / dissimilarity of conditionals? C. R¨ over Implementing mixture approximations December 5, 2014 11 / 31