Why Mixed Effects Models?
Mixed Effects Models Recap/Intro ● Three issues with ANOVA – Multiple random effects – Categorical data – Focus on fixed effects ● What mixed effects models do – Random slopes – Link functions ● Iterative fitting
Problem One: Multiple Random Effects Problem One: Multiple Random Effects ● Most studies sample both subjects and items Subject 1 Subject 2 Subject 1 Subject 2 Monkey Knight Monkey Knight story story story story
Problem One: Crossed Random Effects Problem One: Crossed Random Effects ● Most studies sample both subjects and items – Typically, subjects crossed with items ● Each subject sees a version of each item – May also be only partially crossed ● Each subject sees only some of the items
...or Hierarchical Random Effects ...or Hierarchical Random Effects ● Most studies sample both subjects and items – Typically, subjects crossed with items – May also have one nested within the other ( hierarchical ) ● e.g. autobiographical memory ● How to incorporate this into model?
Problem One: Multiple Random Effects Problem One: Multiple Random Effects ● Why do we care about items, anyway? ● #1: Investigate robustness of effects across items – Concern is that effect could be driven by just 1 or 2 items – might not really be what we thought it was – Psycholinguistics: View is that we studying language too, not just people ● Other areas of psychology have not tended to care about this – Note: Including items in a model doesn't really “confirm ” that the effect is robust across items. It's still possible to get a reliable effect driven by a small number of items. But it allows you investigate how variable the effect is across items and why different items might be differentially influenced.
Problem One: Multiple Random Effects Problem One: Multiple Random Effects ● Why do we care about items? D D C C ● #2: Violations of independence – A BIG ISSUE – Suppose Amélie and Zhenghan see items A & B but Tuan sees items C & D – Likely that Amélie's results are more like Zhenghan's than like Tuan's – But ANOVA assumes observations independent – Even a small amount of dependency can lead to spurious results (Quene & van A A B B den Bergh, 2008) ● Dependency you didn't account for makes the variance look smaller than it actually is
What Constitutes an “Item”? What Constitutes an “Item”? ● Items assumed to be independently sampled sampled from population of relevant items ● 2 related words / sentences not ALL POSSIBLE DISCOURSES independently sampled – “ The coach knew you missed practice.” – “The coach knew that you missed practice.” – Not a coincidence both are in your experiment! ● Should be considered the same item ● But 2 unrelated things can be different items
Problem One: Crossed Random Effects Problem One: Crossed Random Effects ● ANOVA solution Note: not real data or statistical tests – Subjects analysis : Average over multiple items for each subject – Items analysis : F 1 = 18.31, p < .001 Average over multiple subjects for each item ● Two sets of results – Sometime combined with min F' – An approximation of F 2 = 22.10, p < .0001 true min F
Problem One: Crossed Random Effects Problem One: Crossed Random Effects ● Some debate on how Note: not real data or statistical tests accurate min F' is – Scott will admit to not be fully read up on this since I came in after people started switching to mixed effects models F 1 = 18.31, p < .001 ● Somewhat less relevant now that we can use mixed effects models instead F 2 = 22.10, p < .0001
Mixed Effects Models Recap/Intro ● Three issues with ANOVA – Multiple random effects – Categorical data – Focus on fixed effects ● What mixed effects models do – Random slopes – Link functions ● Iterative fitting
Problem Two: Categorical Data ● ANOVA assumes our response is continuous RT: 833 ms ● But, we often want to look at categorical data 'Lightning hit the church.” vs. “The church was hit by lightning.” Choice of Item recalled Region fixated syntactic or not in eye-tracking structure experiment
Problem One: Categorical Data Problem Two: Categorical Data ● Traditional solution: Analyze proportions ● Violates assumptions of ANOVA – Among other issues: ANOVA assumes normal distribution, − which has infinite tails – But proportions are clearly bounded But – Model could predict 0 proportions 1 impossible values like 110%
Problem One: Categorical Data Problem Two: Categorical Data ● Traditional solution: Analyze proportions ● Violates assumptions of ANOVA – Among other issues: ANOVA assumes normal distribution, − which has infinite tails – But proportions are clearly bounded But – Model could predict 0 proportions 1 impossible values like 110%
Problem Two: Categorical Data Problem One: Categorical Data ● Traditional solution: Analyze proportions ● Violates assumptions of ANOVA ● Can lead to: – Spurious effects (Type I error) – Missing a true effects (Type II error)
Problem Two: Categorical Data Problem One: Categorical Data ● Transformations improve the situation but don't solve it – Empirical logit is good (Jaeger, 2008) – Arcsine less so ● Situation is worse for very high or very low proportions (Jaeger, 2008) – .30 to .70 are OK
Problem One: Categorical Data Problem Two: Categorical Data ● Why can't we just use logistic regression? – Predict if each trial's response is in category A or category B ● This is essentially what we will end up doing ● But, if we are looking at things at a trial-by- trial basis... – Need to control for the different items on each trial – Problem One again!
Mixed Effects Models Recap/Intro ● Three issues with ANOVA – Multiple random effects – Categorical data – Focus on fixed effects ● What mixed effects models do – Random slopes – Link functions ● Iterative fitting
Problem Three: Focus on Fixed Effects Problem Three: Focus on Fixed Effects ● ANOVA doesn't characterize differences between subjects or items ● The bird that they spotted was a .... MEAN READING TIME Predictable 283 ms ENDING cardinal cardinal Unpredictable 309 ms 26 ms ● We just have a mean effect pitohui pitohui ● No info. about how much it varies across participants or items
Problem Three: Focus on Fixed Effects Problem Three: Focus on Fixed Effects ● Can try to account for some of this with an ANCOVA – But not typically done – And would have to be done separately for participants and items ( Problem One again) MEAN Predictable 283 ms Unpredictable 309 ms 26 ms
Power of Mixed Effects Models Recap/Intro Power of items subjects analysis! analysis! ● Three issues with ANOVA – Multiple random effects Captain MLM to the rescue! – Categorical data – Focused on fixed effects ● What mixed effects models do – Random slopes – Link functions ● Iterative fitting
Mixed Effects Models to the Rescue! ● ANOVA : Unit of analysis is cell mean ● MLM : Unit of analysis is individual trial !
Mixed Models to the Rescue! ● Look at individual trials ● Model outcome using regression = + + RT Subject RT Item Subject Item Prime? Prime? Semantic categorization: Problem One solved! Problem One solved! Is it a dinosaur?
Mixed Models to the Rescue! ● This means you will need your data formatted differently than you would for an ANOVA – Each trial gets its own line
Mixed Models to the Rescue! ● Is this useful for what we care about? – Stereotypical view of regression is that it's about predicting values – In experimental settings we more typically want to know if Variable X matters ● Yes! We can test individual effects: Do they contribute to the model? – e.g. does priming predict something about RT? = + + RT RT Prime? Prime? Subject Subject Jason Item Jason Item
Mixed Effects Models Recap/Intro ● Three issues with ANOVA – Multiple random effects – Categorical data – Focus on fixed effects ● What mixed effects models do – Random slopes – Link functions ● Iterative fitting
Fixed vs. Random Slopes ● Fixed Slope: Same for all participants/items ● Random Slope: Can vary by participants/items = + + RT RT Prime? Prime? Stego. Laurel Stego. Laurel 26 ms + 88 ms
Fixed vs. Random Slopes ● Fixed Slope: Same for all participants/items ● Random Slope: Can vary by participants/items = + + RT RT Prime? Prime? Dr. L Laurel Dr. L Laurel 26 ms Example: Some items + may show a larger priming effect than others 315 ms
Fixed vs. Random Slopes ● Fixed Slope: Same for all participants/items ● Random Slope: Can vary by participants/items ● Can also test what explains variation = + + RT RT Prime? Prime? Dr. L Laurel Dr. L Laurel 26 ms e.g. Adding lexical frequency + + to the model may account for variation in priming effect Lex.Freq. Lex.Freq. 15 ms 300 ms
Fixed vs. Random Slopes ● Fixed Slope: Same for all participants/items ● Random Slope: Can vary by participants/items ● Can also test what explains variation = + + RT RT Prime? Prime? Dr. L Laurel Dr. L Laurel 26 ms Problem Three Problem Three + + Solved! Solved! Lex.Freq. Lex.Freq. 15 ms 300 ms
Recommend
More recommend
