BayesX: Analysing Bayesian semiparametric regression models Andreas - PowerPoint PPT Presentation

BayesX: Analysing Bayesian semiparametric regression models Andreas Brezger, Thomas Kneib and Stefan Lang Institut f¨ ur Statistik, Universit¨ at M¨ unchen Workshop AG-Bayes, 6. Dezember 2002

A. Brezger, T. Kneib and S. Lang Institut f¨ ur Statistik, Universit¨ at M¨ unchen Outline of the Talk • What is BayesX? • Bayesian semiparametric regression • Example(s) BayesX: Analysing Bayesian semiparametric regression models 1

A. Brezger, T. Kneib and S. Lang Institut f¨ ur Statistik, Universit¨ at M¨ unchen What is BayesX? BayesX is a tool for Bayesian inference via MCMC simulation techniques. available as a Windows (NT, 95, 98, 2000) based application at http://www.stat.uni-muenchen.de/~lang/ BayesX: Analysing Bayesian semiparametric regression models 2

A. Brezger, T. Kneib and S. Lang Institut f¨ ur Statistik, Universit¨ at M¨ unchen Features of the current version • Functions for handling and manipulating data • Functions for handling spatial data • Functions for drawing geographical maps, scatterplots, etc. • Bayesian semiparametric regression • Model selection for DAG’s (by Eva-Maria Fronk) BayesX: Analysing Bayesian semiparametric regression models 3

A. Brezger, T. Kneib and S. Lang Institut f¨ ur Statistik, Universit¨ at M¨ unchen Features of the regression tool • Estimation of any generalized additive model • Response: Gaussian, Poisson, Gamma, Binomial, Multinomial BayesX includes as special cases . . . • Generalized linear models • Generalized additive models • Dynamic or state space models • Varying coefficient models • Mixed models • BYM model for disease mapping BayesX: Analysing Bayesian semiparametric regression models 4

A. Brezger, T. Kneib and S. Lang Institut f¨ ur Statistik, Universit¨ at M¨ unchen Observation models • Distributional and structural assumptions, given covariates and parameters, are based on generalized linear models. • Response: Gaussian, Gamma, Poisson, Binomial, Multinomial • Replace the linear predictor η = z ′ γ by a semiparametric additive predictor η = f 1 ( x 1 ) + · · · + f p ( x p ) + z ′ γ f 1 , ..., f p are unknown functions of the covariates γ parameter vector for fixed effects BayesX: Analysing Bayesian semiparametric regression models 5

A. Brezger, T. Kneib and S. Lang Institut f¨ ur Statistik, Universit¨ at M¨ unchen Extensions Varying coefficient terms η = · · · + f ( x ) z + · · · Surface smoothing η = · · · + f ( x 1 , x 2 ) + · · · BayesX: Analysing Bayesian semiparametric regression models 6

A. Brezger, T. Kneib and S. Lang Institut f¨ ur Statistik, Universit¨ at M¨ unchen Priors for a function f η = f 1 ( x 1 ) + · · · + f p ( x p ) + z ′ γ f = Xβ X design matrix β are unknown parameters η = · · · + Xβ + · · · BayesX: Analysing Bayesian semiparametric regression models 7

A. Brezger, T. Kneib and S. Lang Institut f¨ ur Statistik, Universit¨ at M¨ unchen The general prior β | τ 2 ∝ exp( − 1 2 τ 2 β ′ Kβ ) τ 2 ∼ IG ( a, b ) • K is a penalty matrix that penalizes too rough functions f • structure of K depends on type of covariate and on prior beliefs on smoothness of f • amount of smoothness is controlled by τ 2 BayesX: Analysing Bayesian semiparametric regression models 8

A. Brezger, T. Kneib and S. Lang Institut f¨ ur Statistik, Universit¨ at M¨ unchen Example 1: P-splines (Eilers and Marx, 1996; Lang and Brezger, 2002) f ( x ) = Spline of degree l with equally spaced inner knots ξ 1 , ..., ξ r between x ( min ) and x ( max ) = β 1 B 1 ( x ) + · · · + β r + l +1 B r + l +1 ( x ) B 1 , ..., B r + l +1 B-spline Basis X design matrix with elements X ( i, j ) = B j ( x i ) BayesX: Analysing Bayesian semiparametric regression models 9

A. Brezger, T. Kneib and S. Lang Institut f¨ ur Statistik, Universit¨ at M¨ unchen a) Spline vom Grad 0 b) Spline vom Grad 1 1.5 .5 1 .45 .5 .4 0 .25 .5 .75 1 0 .25 .5 .75 1 c) Spline vom Grad 2 1 .8 .6 .4 0 .25 .5 .75 1 BayesX: Analysing Bayesian semiparametric regression models 10

A. Brezger, T. Kneib and S. Lang Institut f¨ ur Statistik, Universit¨ at M¨ unchen a) Spline vom Grad 0, B-spline Basisfunktion B^0_1 b) Spline vom Grad 0, B-spline Basisfunktion B^0_2 1 1 .5 .5 0 0 0 .25 .5 .75 1 0 .25 .5 .75 1 c) Spline vom Grad 0, B-spline Basisfunktion B^0_3 d) Spline vom Grad 0, B-spline Basisfunktion B^0_4 1 1 .5 .5 0 0 0 .25 .5 .75 1 0 .25 .5 .75 1 BayesX: Analysing Bayesian semiparametric regression models 11

A. Brezger, T. Kneib and S. Lang Institut f¨ ur Statistik, Universit¨ at M¨ unchen a) Spline vom Grad 1, B-spline Basisfunktionen 1 .5 0 -.25 0 .25 .5 .75 1 1.25 b) Spline vom Grad 2, B-spline Basisfunktionen .8 .6 .4 .2 0 -.5 -.25 0 .25 .5 .75 1 1.25 1.5 BayesX: Analysing Bayesian semiparametric regression models 12

A. Brezger, T. Kneib and S. Lang Institut f¨ ur Statistik, Universit¨ at M¨ unchen Example 1: P-splines, frequentist version • relatively large number of inner knots • difference penalty for β 1 , ..., β r + l +1 to penalize too rough functions f • Leads to penalized likelihood estimation m � (∆ k β s ) 2 L = l − λ s = k +1 ∆ k denotes the difference operator of order k . • Problem: Estimation of the smoothing parameter λ . BayesX: Analysing Bayesian semiparametric regression models 13

A. Brezger, T. Kneib and S. Lang Institut f¨ ur Statistik, Universit¨ at M¨ unchen lambda=1000000 lambda=1000000 1.0 1.0 o o o o o o o o o o o o o o o o o o o o o o parameter estimates 0.5 0.5 function estimates 0.0 0.0 -0.5 -0.5 -1.0 -1.0 5 10 15 20 -3 -2 -1 0 1 2 3 parameter number covariate values lambda=100 lambda=100 1.0 1.0 o o o o o parameter estimates 0.5 0.5 o function estimates o o o o 0.0 0.0 o o o o o -0.5 o -0.5 o o o o o o -1.0 -1.0 5 10 15 20 -3 -2 -1 0 1 2 3 parameter number covariate values lambda=0.001 lambda=0.001 1.0 o 1.0 o o o o o parameter estimates 0.5 0.5 function estimates o o o o 0.0 0.0 o o o o -0.5 -0.5 o o o o o -1.0 -1.0 5 10 15 20 -3 -2 -1 0 1 2 3 parameter number covariate values BayesX: Analysing Bayesian semiparametric regression models 14

A. Brezger, T. Kneib and S. Lang Institut f¨ ur Statistik, Universit¨ at M¨ unchen Example 1: P-splines, Bayesian approach • replace difference penalties by their stochastic analogues • smoothness prior for β 1 , ..., β r + l +1 to penalize too rough functions f • use first or second order random walks as smoothness prior: β t = β t − 1 + u t (RW1) β t = 2 β t − 1 − β t − 2 + u t (RW2) u t ∼ N (0 , τ 2 ) BayesX: Analysing Bayesian semiparametric regression models 15

A. Brezger, T. Kneib and S. Lang Institut f¨ ur Statistik, Universit¨ at M¨ unchen RW1: P ( β s | β s − 1 , β s +1 ) ✻ β s +1 s ✻ β s s ❄ τ 2 / 2 β s − 1 s ✲ − 1 0 1 BayesX: Analysing Bayesian semiparametric regression models 16

A. Brezger, T. Kneib and S. Lang Institut f¨ ur Statistik, Universit¨ at M¨ unchen RW2: P ( β s | β s − 1 , β s − 2 ) ✻ τ 2 ✻ β s τ 2 s β s − 1 ❄ s β s − 2 s ✲ − 2 − 1 0 BayesX: Analysing Bayesian semiparametric regression models 17

A. Brezger, T. Kneib and S. Lang Institut f¨ ur Statistik, Universit¨ at M¨ unchen RW2: P ( β s | β s − 1 , β s − 2 , β s +1 , β s +2 ) ✻ β s − 2 , β s +2 s s β s +1 s ✻ β s − 1 s β s s ❄ τ 2 / 6 ✲ -2 -1 0 1 2 BayesX: Analysing Bayesian semiparametric regression models 18

A. Brezger, T. Kneib and S. Lang Institut f¨ ur Statistik, Universit¨ at M¨ unchen Example 2: Markov random fields • Markov random fields (Besag, York, Mollie 1991), e.g.   1 β j , 1 β s | β − s , τ 2 ∼ N � τ 2  N s N s j ∈ ∂ s ∂ s denoting the sites, that are neighbors of site s N s number of neighbors • X 0/1 design matrix BayesX: Analysing Bayesian semiparametric regression models 19

A. Brezger, T. Kneib and S. Lang Institut f¨ ur Statistik, Universit¨ at M¨ unchen BayesX: Analysing Bayesian semiparametric regression models 20

A. Brezger, T. Kneib and S. Lang Institut f¨ ur Statistik, Universit¨ at M¨ unchen Example 3: 2-dimensional surfaces η = · · · + f 1 ( x 1 ) + f 2 ( x 2 ) + f 1 , 2 ( x 1 , x 2 ) + · · · f 1 , 2 = tensor product of one dimensional B-splines m m � � = β ρ,ν B 1 ,ρ ( x 1 ) B 2 ,ν ( x 2 ) . ρ =1 ν =1 spatial smoothness prior fo coefficients β ρ,ν , e.g. 2-dimensional random walks BayesX: Analysing Bayesian semiparametric regression models 21

A. Brezger, T. Kneib and S. Lang Institut f¨ ur Statistik, Universit¨ at M¨ unchen Further examples • random intercepts and slopes • varying coefficient models • time varying seasonal effects BayesX: Analysing Bayesian semiparametric regression models 22

A. Brezger, T. Kneib and S. Lang Institut f¨ ur Statistik, Universit¨ at M¨ unchen Bayesian Inference via MCMC • Draw random numbers from the posterior. • Estimate characteristics of the posterior by their empirical analogue. • Efficiency guaranteed by matrix operations for sparse matrices. Details in Fahrmeir, Lang (2001a,b) Lang and Brezger (2002) BayesX: Analysing Bayesian semiparametric regression models 23