Markov Bases for Logistic Regression Models: A Preliminary Report - PowerPoint PPT Presentation

Markov Bases for Logistic Regression Models: A Preliminary Report Dane Wilburne Illinois Institute of Technology Joint work with: Hisayuki Hara (Niigata University) Loyola University AMS Fall Central Section Meeting Special Session on Algebraic Statistics and its Interactions with Combinatorics, Computation, and Network Science Chicago, IL October 4, 2015 MB for LR October 4, 2015 0 / 9

Poisson Regression 2 × 2 × L tables Poisson Regression x jkl ∼ Po ( λ jkl ) , j = 0 , 1 , k = 0 , 1 , l = 1 , . . . , L log( λ jkl ) = α + β l + γ d j 1 + δ d k 1 � � 1 , j = 1 1 , k = 1 d j d k 1 = otherwise , 1 = 0 , 0 , otherwise l Example ( j , k ) 1 2 . . . L You own a fishing boat, and you would like (0 , 0) ∗ ∗ ∗ . . . to model how many fish are caught with l (0 , 1) ∗ ∗ . . . ∗ fisherman in fair/poor weather ( d j 1 ) on a (1 , 0) ∗ ∗ ∗ . . . weekend/weekday ( d k 1 ). Record data in a (1 , 1) ∗ ∗ . . . ∗ table such as the one on the left. MB for LR October 4, 2015 1 / 9

Poisson Regression 2 × 2 × L tables Poisson Regression x jkl ∼ Po ( λ jkl ) , j = 0 , 1 , k = 0 , 1 , l = 1 , . . . , L log( λ jkl ) = α + β l + γ d j 1 + δ d k 1 � � 1 , j = 1 1 , k = 1 d j d k 1 = otherwise , 1 = 0 , 0 , otherwise Sufficient statistics: Configuration matrix: x +++   1 1 1 1 1 1 1 1 x 1++ . . . . . . . . . . . . 0 0 0 0 1 1 1 1   B = . . . . . . . . . . . .   0 0 1 1 0 0 1 1   . . . . . . . . . . . . x +1+ 1 L 1 L 1 L 1 L . . . . . . . . . . . . � L l =1 ( l · x ++ l ) MB for LR October 4, 2015 1 / 9

Poisson Regression 2 × 2 × . . . × 2 × L tables Markov bases for Poisson Regression Proposition (Hara, W., 2015) There exists a Markov basis for the Poisson regression model for 2 × 2 × L tables containing only degree two moves. l Example (The case L = 3) ( j , k ) 1 2 3 The minimal Markov basis for the (0 , 0) 1 0 0 Poisson regression model for (0 , 1) 0 0 − 1 2 × 2 × 3 tables consists of 33 degree (1 , 0) − 1 0 0 moves of degree two. (1 , 1) 0 0 1 Generalization: One way to generalize this model is to increase the number of dummy variables. In other words, we now consider the Possion regression model for 2 × 2 × . . . × 2 × L tables. � �� � I times MB for LR October 4, 2015 2 / 9

Poisson Regression 2 × 2 × . . . × 2 × L tables Markov bases for Poisson Regression Proposition (Hara, W., 2015) There exists a Markov basis for the Poisson regression model for 2 × 2 × L tables containing only degree two moves. l Example (The case L = 3) ( j , k ) 1 2 3 The minimal Markov basis for the (0 , 0) 1 0 0 Poisson regression model for (0 , 1) 0 0 − 1 2 × 2 × 3 tables consists of 33 degree (1 , 0) − 1 0 0 moves of degree two. (1 , 1) 0 0 1 Proposition (Hara, W. 2015) There exists a Markov basis for the Poisson regression model for 2 × 2 × . . . × 2 × L tables containing only degree two moves. � �� � I times MB for LR October 4, 2015 2 / 9

Partition Identities PPIs Primitive Partition Identities Definition Fix a positive integer n. A partition identity is an identity of the form: a 1 + a 2 + . . . + a k = b 1 + b 2 + . . . + b l , (1) where 0 < a i , b j ≤ n, a i , b j ∈ Z . The quantity k + l is called the degree of the partition identity. We call the partition identity in (1) primitive if there is no proper subidentity a i 1 + a i 2 + . . . + a i r = b j 1 + b j 2 + . . . + b j s , (2) where 1 ≤ r + s ≤ k + l − 1 and we call it homogeneous if k = l. Theorem (Diaconis-Graham-Sturmfels) The degree of any primitive partition identity satisfies k + l ≤ 2 n − 1 . MB for LR October 4, 2015 3 / 9

Partition Identities PPIs Primitive Partition Identities Example 1 + 2 + 3 = 3 + 3 is a non-primitive partition identity with largest part n = 3 and degree 3 + 2 = 5; 1 + 3 + 3 + 3 = 5 + 5 is a primitive partition identity with largest part n = 5 and degree 4 + 2 = 6. Gr ( A ): Graver basis of A ; I A : toric ideal assoc. to A ∈ Z d × n � Gr ( A ) � = I A Observation Let A be the 1 × n integer matrix A = (1 2 n ) . Then, the binomial . . . x a 1 x a 2 · · · x a k − x b 1 x b 2 · · · x b l is a primitive element of I A iff a 1 + a 2 + . . . + a k = b 1 + b 2 + . . . + b l is a ppi, i.e. there is a correspondence between ppi’s and elements of Gr ( A ) . MB for LR October 4, 2015 4 / 9

Partition Identities CHPPIs Color-Homogeneous PPIs Definition A colored partition identity in the colors 1 , . . . , c is an identity of the form: a 1 , 1 + . . . + a 1 , k 1 + a 2 1 + . . . + a 2 , k 2 + a c , 1 + . . . + a c , k c = b 1 , 1 + . . . + b 1 , l 1 + b 2 1 + . . . + b 2 , l 2 + b c , 1 + . . . + b c , l c where 1 ≤ a p , j , b p , j ≤ n p are positive integers for all j, 1 ≤ p ≤ c and some positive integers n 1 , . . . , n c . If k j = l j for all 1 ≤ j ≤ c , then it is called color-homogeneous. A chpi is primitive if there is no proper color-homogeneous subidentity a − , i 1 + . . . + a − , i r = b − , j 1 + . . . + b − , j s with 1 ≤ r + s ≤ k 1 + . . . + k c + l 1 + . . . + l c . The degree is the number of summands k 1 + . . . + k c + l 1 + . . . + l c . MB for LR October 4, 2015 5 / 9

Partition Identities CHPPIs Color-Homogeneous PPIs Example Let c = 2 and let n 1 = n 2 = 3. Then, the following are examples of color-homogeneous partition identities in c = 2 colors with n 1 = n 2 = 3: 1 + 3 = 2 + 2 1 + 3 = 3 + 1 2 + 2 + 3 = 3 + 3 + 1 Theorem (Petrovi´ c) Let n P = max { n i : 1 ≤ i ≤ c } and let n Q = max { n j : 1 ≤ j ≤ c } , j � = P } . Then, any primitive color-homogeneous partition identity satisfies k 1 + . . . + k c + l 1 + . . . + l c ≤ n P + n Q − 2 . Remark Again, there is a correspondence between primitive color-homogeneous partition identitites and Gr ( A ) for some integer matrix A . MB for LR October 4, 2015 6 / 9

Partition Identities CHPPIs Moves of the Poisson model l ( j , k ) 1 2 3 (0 , 0) 0 2 0 (0 , 1) − 3 0 1 (1 , 0) 0 0 − 2 (1 , 1) 2 0 0 We record an entry in the ( l , ( j , k )) cell as l jk , putting positive entries on the left and negative entries on the right, which gives: 2 00 + 2 00 + 3 01 + 1 11 + 1 11 = 1 01 + 1 01 + 1 01 + 3 10 + 3 10 Color according to j : 2 00 + 2 00 + 3 01 + 1 11 + 1 11 = 1 01 + 1 01 + 1 01 + 3 10 + 3 10 Color according to k : 2 00 + 2 00 + 3 01 + 1 11 + 1 11 = 1 01 + 1 01 + 1 01 + 3 10 + 3 10 MB for LR October 4, 2015 7 / 9

Logistic regression 2 × 2 × 2 × L tables Logistic regression � � p 1 jkl = α + β l + γ d j 1 + δ d k log 1 1 − p 1 jkl j = 0 , 1; k = 0 , 1 l = 1 , . . . , L � � 1 , j = 1 1 , k = 1 d j d k 1 = 1 = otherwise , 0 , 0 , otherwise Configuration matrix: Example In the fishing example from earlier, we now � � B 0 Λ( B ) = , think of modeling success/failure outcomes I n × n I n × n instead of counts (i.e. did you meet your quota for the day, or not?) where the where B is the Poisson variables have the same interpretation as configuration (this is before (# of fisherman, weather, called the Lawrence lifting weekend/weekday). of B ). MB for LR October 4, 2015 8 / 9

Logistic regression 2 × 2 × 2 × L tables Markov bases for Logistic Regression The idea is to exploit the connection to generalized CHPIs to prove bounds on the elements of the Gr ( A ), since this corresponds to a Markov basis for I Λ( A ) . Conjecture (Hara, W. 2015) There exists a Markov basis for the logistic regression model for 2 × 2 × 2 × L tables with maximum degree � 6 if L = 2 d ≤ if L ≥ 3 . 6 L − 8 Remark This conjecture has been verified computationally for L ≤ 6 . MB for LR October 4, 2015 9 / 9