A comparison of efficient designs for choices between two options - PowerPoint PPT Presentation

A comparison of efficient designs for choices between two options Heiko Großmann Queen Mary, University of London (joint work with H. Holling, U. Graßhoff and R. Schwabe) mODa 8 Almagro, 7 June 2007

Outline ◮ Background ◮ Relationship between nonlinear choice and linear paired comparison models ◮ Comparison of designs for main effects models ◮ Designs for choice experiments with main effects and interactions 1/25

Choice experiments Where are they used? ◮ Marketing: New product development, consumer research ◮ Health economics, medicine: Preferences for health care interventions ◮ Psychology, behavioral economics: Judgement and decision making, attitude research ◮ Environmental sciences: Valuation of ecosystems, willingness-to-pay for public goods ◮ ... 2/25

Preference measurement 3/25

How can preferences be measured? Decision attributes ◮ Number of windows ◮ Type of floor ◮ Size ◮ Layout 4/25

How can preferences be measured? 5/25

Choice experiments 6/25

Choice experiments 7/25

Choice experiments 8/25

Modeling choices between two options The multinomial logit (MNL) model Choice probabilities e V ( s ) P ( Y ( s , t ) = 1) = e V ( s ) + e V ( t ) where V ( x ) = f ( x ) ⊤ β for every option x 9/25

Modeling choices between two options Information matrix for the MNL model Normalized Fisher information matrix for exact design ξ N with N pairs ( s n , t n ) , n = 1 , . . . , N N M ( ξ N ; β ) = 1 � X ⊤ n ( D n − p n p ⊤ n ) X n N n =1 where ◮ D n = diag( π n , 1 − π n ) , p n = ( π n , 1 − π n ) ⊤ ◮ π n = P ( Y ( s n , t n ) = 1) � f ( s n ) ⊤ � ◮ X n = f ( t n ) ⊤ 10/25

Relation between MNL and linear model for pairs A basis for comparing choice designs ◮ Most optimal designs in the MNL model for pairs have been derived under the assumption that π n = 1 / 2 , or equivalently that β = 0 (Burgess & Street, 2005; Street et al. 2001; Street & Burgess, 2004) ◮ In this case, for every exact design ξ N M ( ξ N ; β ) = 1 4 M ( ξ N ) where M ( ξ N ) = 1 N X ⊤ X is the normalized information matrix in the linear paired comparison model Y ( s , t ) = ( f ( s ) − f ( t )) ⊤ ˜ ˜ β + ε ◮ Implication: Optimal designs for the linear model are also optimal for the MNL model and vice versa, if β = 0 is presumed 11/25

General setting Notation ◮ K factors with levels in X k = { 1 , . . . , v k } , k = 1 , . . . , K ◮ Options: s , t ∈ X 1 × . . . × X K ◮ Approximate design on set of pairs ( s , t ) : � ( s 1 , t 1 ) J � · · · ( s J , t J ) � ξ = where w j = 1 w 1 · · · w J j =1 with information matrix in linear paired comparison model given by J � w j [ f ( s j ) − f ( t j )][ f ( s j ) − f ( t j )] ⊤ M ( ξ ) = j =1 12/25

Main effects model Parametrization ◮ Standard parametrization (effects coding) K ) ⊤ , ˜ β = ( ˜ 1 , . . . , ˜ K ) ⊤ , ˜ β k = (˜ β ( k ) 1 , . . . , ˜ β ( k ) f = ( f ⊤ 1 , . . . , f ⊤ β ⊤ β ⊤ v k − 1 ) ⊤   x k th unit vector of length v k − 1 x k ∈ { 1 , . . . , v k − 1 }   f k ( x k ) =  − 1 v k − 1 x k = v k   ◮ Number of parameters: p = � K k =1 v k − K 13/25

Optimality results: main effects model Theorem (Graßhoff et al., 2004) For K factors with levels v 1 , . . . , v K an approximate design ξ ∗ is D -optimal, if and only if   M 1 0 ... M ( ξ ∗ ) =     0 M K where for k = 1 , . . . , K the ( v k − 1) × ( v k − 1) matrix M k is given by 2 1 1   · · · · · · . ... ... .   . 1   2   . . ... ... ... M k =   . .   . . v k − 1    .  ... ... .   . 1   1 1 2 · · · · · · 14/25

Optimal exact designs: main effects model Available results Same number of levels for all factors, that is v k = v , k = 1 , . . . , K ◮ v = 2 : Foldover construction based on regular fractions of resolution III or higher (Street & Burgess, 2004) Pairs: 2 K − m or 2 K − m − 1 where m is largest number for which fraction of required resolution exists ◮ v ≥ 2 : Hadamard matrix construction (Graßhoff et al., 2004) Pairs: H K v ( v − 1) / 2 where H K ≤ K + 3 15/25

Optimal exact designs: main effects model Available results Different numbers of levels allowed ◮ Method based on ‘difference vectors’ (Burgess & Street, 2005) Pairs: Multiple of � K k =1 v k ◮ Construction using asymmetric orthogonal arrays of strength 2 (Graßhoff et al., 2004) Pairs: Size N of smallest OA( N ; m 1 , . . . , m K ; 2) where m k is a multiple of v k ( v k − 1) / 2 if v k is odd and a multiple of v k ( v k − 1) if v k is even 16/25

Examples Factors with different numbers of levels ◮ K = 7 , v 1 = . . . = v 6 = 3 , v 7 = 4 Burgess & Street (2005): 3 6 × 4 × 3 = 8748 pairs Grasshoff et al. (2004): 18 pairs ◮ K 1 ≤ 11 two-level and K 2 ≤ 12 three-level factors, K 1 + K 2 = K Burgess & Street (2005): 2 K 1 × 3 K 2 pairs Grasshoff et al. (2004): 36 pairs 17/25

Design comparison for main effects model Remarks ◮ If all factors have v = 2 levels, the Hadamard matrix construction and the foldover construction yield designs of the same size (for K ≤ 8 ) provided that whenever possible the m defining contrasts needed for the latter construction have even wordlength ◮ For factors with different numbers of levels, the method of Grasshoff et al. (2004) yields smaller optimal designs than the construction in Burgess & Street (2005) ◮ The method of Burgess & Street (2005) is more widely applicable but the reduction achieved is often not very large when compared to the product-type optimal design with 1 / 2 � K k =1 N k pairs, where N k = v k ( v k − 1) for v k even and N k = v k ( v k − 1) / 2 for v k odd 18/25

Model including main effects and interactions Parametrization ◮ Consider only the case v k = 2 , k = 1 , . . . , K ◮ Standard parametrization (effects coding) f ( x ) = ( g 1 ( x 1 ) , . . . , g K ( x K ) , g 1 ( x 1 ) g 2 ( x 2 ) , . . . , g K − 1 ( x K − 1 ) g K ( x K )) ⊤ where g k (1) = 1 and g k (2) = − 1 for k = 1 , . . . , K ◮ Number of parameters: p = K + K ( K − 1) / 2 19/25

Optimality results: main and interaction effects Theorem (van Berkum, 1987; Street et al., 2001) ◮ If K is odd, then the approximate design ξ ∗ which is uniform on the set of all pairs differing in exactly d ∗ = ( K + 1) / 2 factors is D -optimal with information matrix M ( ξ ∗ ) = 4 d ∗ /K I p ◮ If K is even, then the approximate design ξ ∗ which is uniform on the set of all pairs differing in exactly d ∗ = K/ 2 or d ∗ + 1 factors is D -optimal with information matrix M ( ξ ∗ ) = 4( d ∗ + 1) / ( K + 1) I p ◮ Generalization when common number of levels is larger than 2 : Graßhoff et al. (2003) 20/25

Performance of uniform designs on sets of pairs which differ in fixed number of factors D -efficiencies 3 4 5 6 7 8 9 10 K d ∗ 2 2 3 3 4 4 5 5 eff D (¯ ξ d ∗ ) 1 . 000 0 . 990 1 . 000 0 . 997 1 . 000 0 . 999 1 . 000 0 . 999 eff D (¯ ξ d ∗ − 1 ) 0 . 707 0 . 632 0 . 874 0 . 816 0 . 931 0 . 891 0 . 956 0 . 928 eff D (¯ ξ d ∗ +1 ) 0 . 000 0 . 980 0 . 840 0 . 995 0 . 922 0 . 998 0 . 953 0 . 999 Note: For every d the uniform design on the set of pairs which differ in exactly d factors is denoted by ¯ ξ d . The information matrix M (¯ ξ d ) is diagonal 21/25

Exact designs: main and interaction effects Available results ◮ Method based on two-level fractional factorials of resolution V or higher in K factors and ‘difference vectors’ (Street & Burgess, 2004) Pairs: Multiple of size of the fractional factorial ◮ Construction for pairs which differ in exactly d factors using BIBDs, Hadamard matrices and two-level fractional factorials of resolution III or higher in K − d factors (Großmann et al., 2007) Pairs: bmn where n ≤ d + 3 , m is the size of a fractional factorial in K − d factors and b the number of blocks of a BIBD( K, b, r, d, λ ) 22/25

Design comparison: main and interaction effects GSG07 SB04 BIBD( K, b, r, d, λ ) Fact. Pairs D -eff. Pairs D -eff. K d n 2 1 3 2 2 BIBD(3 , 3 , 2 , 2 , 1) 12 1 . 000 12 1 . 000 2 2 4 2 2 BIBD(4 , 6 , 3 , 2 , 1) 48 0 . 990 48 0 . 990 2 2 5 3 4 BIBD(5 , 10 , 6 , 3 , 3) 160 1 . 000 160 1 . 000 2 3 − 1 6 3 4 BIBD(6 , 10 , 5 , 3 , 2) 160 0 . 997 224 1 . 000 III 2 3 − 1 7 4 4 BIBD(7 , 7 , 4 , 4 , 2) 112 1 . 000 224 1 . 000 III 2 4 − 1 8 4 4 BIBD(8 , 14 , 7 , 4 , 3) 448 0 . 999 1056 0 . 999 IV 23/25

Concluding remarks Main points ◮ When the parameter vector in a multinomial logit model is presumed to be equal to zero the optimal design problem for choices between two options is equivalent to the corresponding problem for linear paired comparisons ◮ Optimal and efficient linear paired comparison designs often require a considerably smaller number of pairs than choice designs available in the literature ◮ Construction methods developed by Burgess and Street have a wider range of applicability than corresponding methods for linear paired comparisons ◮ Designs considered here can be adapted to other contexts such as two-colour microarray experiments 24/25