Main Effects vs. Simple Effects Scott Fraundorf MLM Reading Group - PowerPoint PPT Presentation

Main Effects vs. Simple Effects Scott Fraundorf MLM Reading Group April 7th, 2011 If you want to talk 10 10 about main effects, 9 need to distinguish 9 true “main effects”... 8 8 7 7 6 6 ...from their 5 5 insidious cousin that 4 4 may be 3 3 masquerading as “main effect” in your 2 2 model 1 1 0 0

Outline  The Problem  Recap of Coding  Parameter Testing  Simple Effects & Main Effects  More Detailed Explanation  How to Do Contrast Coding  Continuous Predictors

1 0.9 0.8 0.7 0.6 Presentational 0.5 Contrastive 0.4 0.3 Prototypical psychology 0.2 study: 2 x 2 0.1 design 0 Contrast Unmentioned

Example Study  Study easy and difficult word pairs – VIKING—HELMET ( related and thus easy ) – VIKING—COLLEGE ( unrelated and thus hard )  Do cued recall task: – VIKING---?????  During test phase, told if an opponent supposedly got the item correct or incorrect

Easy items are remembered INTERACTION! better if the opponent supposedly got them right. 1 Hard items are remembered better if the opponent got them 0.9 wrong. (i.e., performance best in the 0.8 MISMATCH conditions) 0.7 Effect of feedback depends on item type (INTERACTION). 0.6 Accuracy Opponent Correct 0.5 Opponent Incorrect 0.4 0.3 0.2 0.1 0 Easy Items Hard Items

INTERACTION! 1 0.9 0.8 0.7 0.6 Accuracy Opponent Correct 0.5 Opponent Incorrect 0.4 0.3 0.2 0.1 Overall, the related (easy) 0 items are also remembered Easy Items Hard Items better than the unrelated (hard) items MAIN EFFECT

INTERACTION! 1 0.9 0.8 0.7 0.6 Accuracy Opponent Correct 0.5 Opponent Incorrect 0.4 MAIN EFFECT (OR LACK THEREOF) 0.3 No consistent effect of 0.2 opponent feedback. 0.1 0 Easy Items Hard Items MAIN EFFECT

The Problem ANOVA WORLD MLM WORLD ● Get test of interaction ● Not in Kansas anymore! ● What to do? ● And of 2 main effects

The Problem  Modeling our outcome variable in a regression equation β 2 X 2 + β 12 X 1 X 2 + ... β 0 + β 1 X 1 + Y = β 0  Need to code categorical variables into numerical ones  Consequences for how you interpret hypothesis tests R's secret decoder wheel

Outline  The Problem  Recap of Coding  Parameter Testing  Simple Effects & Main Effects  More Detailed Explanation  How to Do Contrast Coding  Continuous Predictors

ITEM TYPE OPPONENT One level is 1 FEEDBACK Other level is 0 Correct : 0 Related : 0 Incorrect : 1 Unrelated : 1 DUMMY CODING a/k/a TREATMENT CODING (R's default)

Predictor with >2 levels: get more dummy-coded variables OPPONENT (B) ITEM TYPE OPPONENT (A) Didn't See : 0 Didn't See : 0 Correct : 0 Correct : 1 Related : 0 Incorrect : 1 Incorrect : 0 Unrelated : 1 DUMMY CODING a/k/a TREATMENT CODING (R's default)

ITEM TYPE OPPONENT One level is 1 FEEDBACK Other level is 0 Correct : 0 Related : 0 Incorrect : 1 Unrelated : 1 DUMMY CODING CONTRAST CODING One level is ITEM TYPE OPPONENT positive FEEDBACK Other level is Correct : -0.5 Related : -0.5 negative Incorrect : 0.5 Unrelated : 0.5

Outline  The Problem  Recap of Coding  Parameter Testing  Simple Effects & Main Effects  More Detailed Explanation  How to Do Contrast Coding  Continuous Predictors

Testing a Parameter How to tell if the opponent's feedback is related to memory? (e.g. possible main effect: you just try harder when someone else got the item wrong) β 2 X 2 + ... β 0 + β 1 X 1 + Y = β 0 Item Type Feedback 0 = Related 0 = Correct 1 = Unrelated 1 = Incorrect

Testing a Parameter Compare when feedback = 0... β 2 X 2 + ... β 0 + β 1 0 + Y = β 0 Item Type Feedback 0 = Related 0 = Correct 1 = Unrelated 1 = Incorrect

Testing a Parameter … to when feedback = 1 β 2 X 2 + ... β 0 + β 1 1 + Y = β 0 Item Type Feedback 0 = Related 0 = Correct 1 = Unrelated 1 = Incorrect

Testing a Parameter β 1 : “The effect of changing feedback, while holding item type constant” But, we know there's β 2 X 2 + ... β 0 + β 1 X 1 + Y = β 0 an interaction … so it will matter what value we hold item type constant at! Item Type Feedback 0 = Related 0 = Correct 1 = Unrelated 1 = Incorrect

Testing a Parameter Hold other variable at: 0 β 2 X 2 + ... β 0 + β 1 X 1 + Y = β 0 With dummy coding, item type = 0 represents the Probe Feedback RELATED condition. 0 = Correct Testing effect of 1 = Incorrect feedback in just the RELATED condition.

INTERACTION! 1 0.9 Hold other variable at: 0 0.8 0.7 0.6 Accuracy Opponent Correct 0.5 Opponent Incorrect With dummy coding, 0.4 item type = 0 0.3 represents the Here, we see RELATED condition. Opponent 0.2 Incorrect > 0.1 Testing effect of Opponent feedback in just the Correct 0 Easy Items RELATED condition. Hard Items But, this test only reflects HALF of the graph!

INTERACTION! 1 0.9 0.8 0.7 0.6 Accuracy Opponent Correct 0.5 Opponent Incorrect 0.4 0.3 0.2 0.1 0 Feedback effect is Easy Items Hard Items But, this test only reflects different in the other half. HALF of the graph! Misleading!

Outline  The Problem  Recap of Coding  Parameter Testing  Simple Effects & Main Effects  More Detailed Explanation  How to Do Contrast Coding  Continuous Predictors

FEEDBACK ITEM TYPE Related : 0 Correct : 0 Unrelated : 1 Incorrect : 1 DUMMY CODING What is effect of the Feedback variable? R holds Item Type at 0. Only reflects Related condition. Problem: Effect of Feedback depends on the Item Type. (i.e., Simple Effect there's an INTERACTION)

Main Effect What is effect of the Feedback variable? R holds Item Type at 0 . Averaged between Test now uses 2 conditions. information from both Item Types in testing Feedback. (No main effect here.) CONTRAST CODING FEEDBACK ITEM TYPE Related : -0.5 Correct : -0.5 Unrelated : 0.5 Incorrect : 0.5

 Dummy Coding -> Simple Effects  Consider only one level of predictor X 2 in testing predictor X 1  Contrast Coding -> Main Effects  Consider all levels of predictor X 2 in testing predictor X 1  Both are legitimate statistical tests, but they test different things – Simple effects may be appropriate if you WANT to only test at one level of predictor X 2 – e.g. that level is the baseline ( “opponent didn't see” condition?) – Just make sure that your tests are testing what you say they are!

Some Other Notes... INTERACTION! ● Coding differences do not affect the test of the interaction 1 0.9 ● Coding only changes 0.8 Accuracy 0.7 the simple/main 0.6 Opponent effect terms Correct 0.5 Opponent 0.4 ● Also doesn't change Incorrect 0.3 overall fit of the model 0.2 0.1 0 Hard Items Easy Items

Some Other Notes... ● If NO interaction, simple effects and main INTERACTION! effects are the same ● X 2 is irrelevant to X 1 1 effect 0.9 0.8 ● But note that even if Accuracy 0.7 interaction isn't reliable 0.6 Opponent Correct 0.5 at α = .05, there can be Opponent 0.4 a numerical interaction Incorrect 0.3 0.2 ● Would still be some 0.1 0 difference between Hard Items simple effects & Easy Items main effects

Outline  The Problem  Recap of Coding  Parameter Testing  Simple Effects & Main Effects  More Detailed Explanation  How to Do Contrast Coding  Continuous Predictors

Dummy Coding  Y = β 0 + β 1 X 1 + β 2 X 2 + β 12 X 1 X 2 (+ random effects, error)  X 1 = 0 if related, 1 if unrelated  X 2 = 0 if opponent right, 1 if opponent wrong  Results: – Related, Right: β 0 + β 1 X 1 + β 2 X 2 + β 12 X 1 X 2 – = β 0 + β 1 (0) + β 2 (0) + β 12 (0)(0) (substituting in 0s for X 1 and X 2 )

Dummy Coding  Y = β 0 + β 1 X 1 + β 2 X 2 + β 12 X 1 X 2 (+ random effects, error)  X 1 = 0 if related, 1 if unrelated  X 2 = 0 if opponent right, 1 if opponent wrong  Results: – Related, Right: β 0 + β 1 X 1 + β 2 X 2 + β 12 X 1 X 2 – = β 0 + β 1 (0) + β 2 (0) + β 12 (0)(0) – Most of this is 0 and drops out

Dummy Coding  Y = β 0 + β 1 X 1 + β 2 X 2 + β 12 X 1 X 2 (+ random effects, error)  X 1 = 0 if related, 1 if unrelated  X 2 = 0 if opponent right, 1 if opnonent wrong  Results: If we create the – Related, Right: β 0 equation for all 4 conditions... – Unrelated, Right: β 0 + β 1 – Related, Wrong: β 0 + β 2 – Unrelated, Wrong: β 0 + β 1 + β 2 + β 12

Dummy Coding  Y = β 0 + β 1 X 1 + β 2 X 2 + β 12 X 1 X 2 (+ random effects, error)  X 1 = 0 if related, 1 if unrelated  X 2 = 0 if opponent right, 1 if opnonent wrong We see that, here,  Results: β 1 = Difference – Related, Right: β 0 between Related, Right – Unrelated, Right: β 0 + β 1 and Unrelated, Right – Related, Wrong: β 0 + β 2 – Unrelated, Wrong: β 0 + β 1 + β 2 + β 12