Bayesian inference for logistic models using Polya-gamma latent - PowerPoint PPT Presentation

Bayesian inference for logistic models using Polya-gamma latent variables Nicholas G. Polson, James G. Scott, Jesse Windle (Slides based on slides by James G. Scott) 1

Modeling binary data Age AgeGroup Race Completed InsuranceType Location PracticeType 515 21 18to26 Black 0 Military Odenton FamilyPractice 423 21 18to26 Black 0 PrivatePayer Odenton FamilyPractice 388 17 11to17 White 0 PrivatePayer Odenton Pediatric 6 11 11to17 Black 0 Medicaid Odenton Pediatric 1104 19 18to26 Black 0 Medicaid Bayview Pediatric 1412 19 18to26 Black 0 Medicaid JohnsHopkins OBGYN 1354 24 18to26 White 0 PrivatePayer JohnsHopkins OBGYN 318 18 18to26 Black 1 Military Odenton FamilyPractice 768 24 18to26 White 1 PrivatePayer Odenton OBGYN 29 13 11to17 Other/Unknown 0 PrivatePayer Odenton FamilyPractice 1173 14 11to17 Hispanic 0 PrivatePayer Bayview Pediatric 799 24 18to26 White 0 PrivatePayer Odenton OBGYN 633 24 18to26 White 1 PrivatePayer WhiteMarsh OBGYN 111 13 11to17 Other/Unknown 0 Medicaid Odenton Pediatric 69 15 11to17 Black 0 PrivatePayer Odenton FamilyPractice 559 12 11to17 Black 0 Military Odenton Pediatric 1289 26 18to26 White 1 HospitalBased Bayview OBGYN 1127 18 11to17 White 0 Medicaid Bayview Pediatric 1250 18 11to17 Black 0 PrivatePayer Bayview Pediatric 1098 15 11to17 White 1 Medicaid Bayview Pediatric 378 12 11to17 White 1 Military Odenton FamilyPractice 702 26 18to26 White 0 PrivatePayer WhiteMarsh OBGYN .... Idea: p ( y | x ) = f ( β ⊤ x ) 2

Modeling binary data Age AgeGroup Race Completed InsuranceType Location PracticeType 515 21 18to26 Black 0 Military Odenton FamilyPractice 423 21 18to26 Black 0 PrivatePayer Odenton FamilyPractice 388 17 11to17 White 0 PrivatePayer Odenton Pediatric 6 11 11to17 Black 0 Medicaid Odenton Pediatric 1104 19 18to26 Black 0 Medicaid Bayview Pediatric 1412 19 18to26 Black 0 Medicaid JohnsHopkins OBGYN 1354 24 18to26 White 0 PrivatePayer JohnsHopkins OBGYN 318 18 18to26 Black 1 Military Odenton FamilyPractice 768 24 18to26 White 1 PrivatePayer Odenton OBGYN 29 13 11to17 Other/Unknown 0 PrivatePayer Odenton FamilyPractice 1173 14 11to17 Hispanic 0 PrivatePayer Bayview Pediatric 799 24 18to26 White 0 PrivatePayer Odenton OBGYN 633 24 18to26 White 1 PrivatePayer WhiteMarsh OBGYN 111 13 11to17 Other/Unknown 0 Medicaid Odenton Pediatric 69 15 11to17 Black 0 PrivatePayer Odenton FamilyPractice 559 12 11to17 Black 0 Military Odenton Pediatric 1289 26 18to26 White 1 HospitalBased Bayview OBGYN 1127 18 11to17 White 0 Medicaid Bayview Pediatric 1250 18 11to17 Black 0 PrivatePayer Bayview Pediatric 1098 15 11to17 White 1 Medicaid Bayview Pediatric 378 12 11to17 White 1 Military Odenton FamilyPractice 702 26 18to26 White 0 PrivatePayer WhiteMarsh OBGYN .... Idea: p ( y | x ) = f ( β ⊤ x ) Non-Bayesian: logistic regression Bayesian: probit regression 2

Why probit? • Simple auxiliary variable trick (Albert and Chib) 3

�� N N i β ) } 1 − y i · { 1 − Φ( x T � � { Φ( x T i β ) } y i L ( β ) = L i ( β ) = �� i =1 i =1 �� 1 � exp { − ( z i − x T i β ) 2 / 2 } dz i L i ( β ) = √ 2 � A i � ( −∞ , 0) , y i = 0 A i = (0 , ∞ ) , y i = 1 . -2 0 2 4 -2 0 2 4 4

�� Auxiliary variables �� p ( β | Y ) p ( β ) L ( β ) ∝ N � � � ( z i ; x T = p ( β ) i β , 1) dz i A i i =1 n N � � � � � � ( z i ; x T = z ∈ A i p ( β ) i β , 1) dz R n I i =1 i =1 � R n p ( β , z | Y ) dz ∝ �� 5

�� Auxiliary variables �� p ( β | Y ) p ( β ) L ( β ) ∝ N � � � ( z i ; x T = p ( β ) i β, 1) dz i A i i =1 n N � � � � � � ( z i ; x T = z ∈ A i p ( β ) i β, 1) dz R n I i =1 i =1 � R n p ( β, z | Y ) dz ∝ Gibbs sampling: alternate between p ( β | z , Y ) (Gaussian) and �� p ( z | β, Y ) (Truncated Gaussian) �� 5

�� Auxiliary variables �� p ( β | Y ) p ( β ) L ( β ) ∝ N � � � ( z i ; x T = p ( β ) i β, 1) dz i A i i =1 n N � � � � � � ( z i ; x T = z ∈ A i p ( β ) i β, 1) dz R n I i =1 i =1 � R n p ( β, z | Y ) dz ∝ Gibbs sampling: alternate between p ( β | z , Y ) (Gaussian) and �� p ( z | β, Y ) (Truncated Gaussian) �� Similar ideas for models involving fancier likelihoods (binomial, negative binomial etc.) 5

Auxiliary variable representation for logistic �� likelihood? N { exp( x T i β ) } y i � p ( β | Y ) ∝ p ( β ) · 1 + exp( x T i β ) i =1 � ? = p ( β, z | Y ) dz . 6

�� Polya Gamma distribution �� X ∼ �� ( b, c ) �� ∞ 1 g k D � X = ( k − 1 / 2) 2 + c 2 / (4 π 2 ) 2 π 2 k =1 iid �� ( b, 1) g k ∼ �� 7

Bayesian inference for logistic models using Polya-gamma latent - PowerPoint PPT Presentation

Bayesian inference for logistic models using Polya-gamma latent variables Nicholas G. Polson, James G. Scott, Jesse Windle (Slides based on slides by James G. Scott) 1 Modeling binary data Age AgeGroup Race Completed InsuranceType

Approximate Bayesian inference for latent Gaussian models avard Rue 1 H Department of

BAYESIAN CALIBRATION OF COMPUTER MODELS Bayesian inference & Markov chain Monte Carlo

Bayesian Hierarchical Models for parameter inference with missing

Bayesian Hierarchical Models for parameter inference with missing

Case Study: Approximate Bayesian Inference for Latent Gaussian Models by Using Integrated Nested

CS440/ECE448 Lecture 15: Bayesian Inference and Bayesian Learning Slides by Svetlana Lazebnik,

Logistic mixed models for DIF IRT models can be regarded as logistic mixed models (e.g., Adams,

Analytics, Inference and Computation in Cosmology: Exercises on Bayesian Inference Roberto

Bayesian nonparametric inference for diffusion models with discrete sampling Delft University of

Chapter 3 Parametric Models and Methods Models: Weibull ( includes the exponential model)

Surrogate models and reduction methods for UQ and inference in large-scale models Olivier Le

CS 730/830: Intro AI Bayesian Networks Approx. Inference Exact Inference Wheeler Ruml (UNH)

The role of mechanistic models in Bayesian inference Dan Cornford 1 , Alexis Boukouvalas, Yuan

Entropy Inference Based on An Objective Bayesian Approach for Upper Record Values Having the

Efficient Bayesian inference for Copula Gaussian graphical models A. Mohammadi, F. Abegaz and E.

Machine Learning: Foundations Lecturer: Yishay Mansour Lecture 2 Bayesian Inference Kfir Bar

Meta-Bayesian Analysis A Bayesian decision-theoretic analysis of Bayesian inference under model

EST5104 Bayesian Inference EST5803 Advanced Bayesian Inference Ricardo Ehlers ehlers@icmc.usp.br

Inference in Bayesian networks Chapter 14.45 Chapter 14.45 1 Outline Exact inference

Scalable Bayesian inference of dendritic voltage via spatiotemporal recurrent state space models

Bayesian inference & process convolution models Dave Higdon, Statistical Sciences Group, LANL

Exact inference (Ch. 14) Bayesian Network A Bayesian network (Bayes net) is: (1) a directed

A Course in Applied Econometrics 1. Introduction Lecture 13 2. Basics Bayesian Inference

Hierarchical Bayesian ARX models for robust inference Johan Dahlin, Fredrik Lindsten, Thomas B.

Bayesian inference for logistic models using Polya-gamma latent - PowerPoint PPT Presentation

Bayesian inference for logistic models using Polya-gamma latent variables Nicholas G. Polson, James G. Scott, Jesse Windle (Slides based on slides by James G. Scott) 1 Modeling binary data Age AgeGroup Race Completed InsuranceType

Approximate Bayesian inference for latent Gaussian models avard Rue 1 H Department of

BAYESIAN CALIBRATION OF COMPUTER MODELS Bayesian inference &amp; Markov chain Monte Carlo

Bayesian Hierarchical Models for parameter inference with missing

Bayesian Hierarchical Models for parameter inference with missing

Case Study: Approximate Bayesian Inference for Latent Gaussian Models by Using Integrated Nested

CS440/ECE448 Lecture 15: Bayesian Inference and Bayesian Learning Slides by Svetlana Lazebnik,

Logistic mixed models for DIF IRT models can be regarded as logistic mixed models (e.g., Adams,

Analytics, Inference and Computation in Cosmology: Exercises on Bayesian Inference Roberto

Bayesian nonparametric inference for diffusion models with discrete sampling Delft University of

Chapter 3 Parametric Models and Methods Models: Weibull ( includes the exponential model)

Surrogate models and reduction methods for UQ and inference in large-scale models Olivier Le

CS 730/830: Intro AI Bayesian Networks Approx. Inference Exact Inference Wheeler Ruml (UNH)

The role of mechanistic models in Bayesian inference Dan Cornford 1 , Alexis Boukouvalas, Yuan

Entropy Inference Based on An Objective Bayesian Approach for Upper Record Values Having the

Efficient Bayesian inference for Copula Gaussian graphical models A. Mohammadi, F. Abegaz and E.

Machine Learning: Foundations Lecturer: Yishay Mansour Lecture 2 Bayesian Inference Kfir Bar

Meta-Bayesian Analysis A Bayesian decision-theoretic analysis of Bayesian inference under model

EST5104 Bayesian Inference EST5803 Advanced Bayesian Inference Ricardo Ehlers ehlers@icmc.usp.br

Inference in Bayesian networks Chapter 14.45 Chapter 14.45 1 Outline Exact inference

Scalable Bayesian inference of dendritic voltage via spatiotemporal recurrent state space models

Bayesian inference &amp; process convolution models Dave Higdon, Statistical Sciences Group, LANL

Exact inference (Ch. 14) Bayesian Network A Bayesian network (Bayes net) is: (1) a directed

A Course in Applied Econometrics 1. Introduction Lecture 13 2. Basics Bayesian Inference

Hierarchical Bayesian ARX models for robust inference Johan Dahlin, Fredrik Lindsten, Thomas B.

BAYESIAN CALIBRATION OF COMPUTER MODELS Bayesian inference & Markov chain Monte Carlo

Bayesian inference & process convolution models Dave Higdon, Statistical Sciences Group, LANL