course business
play

Course Business l New datasets on CourseWeb for Week 5 l But first, - PowerPoint PPT Presentation

Sep 30, 2023 •45 likes •1.17k views

Course Business l New datasets on CourseWeb for Week 5 l But first, we will finish last weeks math.csv l Midterm assignment: Review a journal article in your area that uses mixed-effects models l Goals: l Practice interpreting mixed-effects

  1. New Dataset • tutor.csv • Effects of computer math tutor on FinalMathScore s • 25 students in each of 10 classrooms in each of 20 schools • Once you get the dataframe loaded into R, you might notice a problem. See if you can identify it and fix it. (Work with your table!) • R thinks ClassroomID is a numeric variable tutor$ClassroomID <- • as.factor(tutor$ClassroomID)

  2. Multiple Random Effects LEVEL 3 School School Sampled SCHOOLS 1 2 LEVEL 2 Ms. Mr. Ms. Ms. Sampled Wagner’s Fulton’s Green’s Cornell’s Class Class Class Class CLASSROOMS LEVEL 1 Student Student Student Student Sampled STUDENTS 1 2 3 4 • Let’s do an intervention: Hours of use of math tutoring software • Which level(s) of the model could this be at?

  3. Multiple Random Effects LEVEL 3 School School Sampled SCHOOLS 1 2 If use of the tutor characterizes a whole school LEVEL 2 Ms. Mr. Ms. Ms. Sampled Wagner’s Fulton’s Green’s Cornell’s Class Class Class Class CLASSROOMS LEVEL 1 Student Student Student Student Sampled STUDENTS 1 2 3 4 • Let’s do an intervention: Hours of use of math tutoring software • Which level(s) of the model could this be at?

  4. Multiple Random Effects LEVEL 3 School School Sampled SCHOOLS 1 2 LEVEL 2 Ms. Mr. Ms. Ms. Sampled Wagner’s Fulton’s Green’s Cornell’s Class Class Class Class CLASSROOMS If classrooms within a school vary in tutor use, but consistent within a classroom LEVEL 1 Student Student Student Student Sampled STUDENTS 1 2 3 4 • Let’s do an intervention: Hours of use of math tutoring software • Which level(s) of the model could this be at?

  5. Multiple Random Effects LEVEL 3 School School Sampled SCHOOLS 1 2 LEVEL 2 Ms. Mr. Ms. Ms. Sampled Wagner’s Fulton’s Green’s Cornell’s Class Class Class Class CLASSROOMS LEVEL 1 Student Student Student Student Sampled STUDENTS 1 2 3 4 If students within a classroom varied in their tutor usage • Let’s do an intervention: Hours of use of math tutoring software • Which level(s) of the model could this be at?

  6. Multiple Random Effects LEVEL 3 School School Sampled SCHOOLS 1 2 LEVEL 2 Ms. Mr. Ms. Ms. Sampled Wagner’s Fulton’s Green’s Cornell’s Class Class Class Class CLASSROOMS LEVEL 1 Student Student Student Student Sampled STUDENTS 1 2 3 4 • Can you find this out from R?

  7. Multiple Random Effects LEVEL 3 School School Sampled SCHOOLS 1 2 LEVEL 2 Ms. Mr. Ms. Ms. Sampled Wagner’s Fulton’s Green’s Cornell’s Class Class Class Class CLASSROOMS If classrooms within a school vary in tutor use, but consistent within a classroom LEVEL 1 Student Student Student Student Sampled STUDENTS 1 2 3 4 • Can you find this out from R? xtabs( ~ ClassroomID + TutorHours, data=tutor) •

  8. Multiple Random Effects LEVEL 3 School School Sampled SCHOOLS 1 2 LEVEL 2 Ms. Mr. Ms. Ms. Sampled Wagner’s Fulton’s Green’s Cornell’s Class Class Class Class CLASSROOMS If classrooms within a school vary in tutor use, but consistent within a classroom LEVEL 1 Student Student Student Student Sampled STUDENTS 1 2 3 4 • Try running a model to look at the effect of TutorHours on FinalMathScore • Hint: Don’t forget to account for the clustering! • Hint 2: You will need 2 random intercepts

  9. Multiple Random Effects LEVEL 3 School School Sampled SCHOOLS 1 2 LEVEL 2 Ms. Mr. Ms. Ms. Sampled Wagner’s Fulton’s Green’s Cornell’s Class Class Class Class CLASSROOMS If classrooms within a school vary in tutor use, but consistent within a classroom LEVEL 1 Student Student Student Student Sampled STUDENTS 1 2 3 4 model.Intercepts <- lmer(FinalMathScore ~ 1 + • TutorHours + (1|ClassroomID) + (1|School), data=tutor)

  10. Multiple Random Effects LEVEL 3 School School Sampled SCHOOLS 1 2 LEVEL 2 Ms. Mr. Ms. Ms. Sampled Wagner’s Fulton’s Green’s Cornell’s Class Class Class Class CLASSROOMS If classrooms within a school vary in tutor use, but consistent within a classroom LEVEL 1 Student Student Student Student Sampled STUDENTS 1 2 3 4 • Which do you think is better? • Teachers are allowed to choose how much they use the tutor in their classroom • We randomly assign a specific # of tutor hours

  11. Multiple Random Effects LEVEL 3 School School Sampled SCHOOLS 1 2 LEVEL 2 Ms. Mr. Ms. Ms. Sampled Wagner’s Fulton’s Green’s Cornell’s Class Class Class Class CLASSROOMS If classrooms within a school vary in tutor use, but consistent within a classroom LEVEL 1 Student Student Student Student Sampled STUDENTS 1 2 3 4 • Random assignment allows stronger claims about causalit y • But, model is implemented the same either way • Just affects how we interpret the results

  12. Introduction to Random Slopes • Right now, our assumption is that schools differ in their baseline math score (intercept) • What other ways might schools differ? • Some schools might use the tutor more effectively than others • Fidelity of implementation is a major concern in education research!

  13. Introduction to Random Slopes • Overall relationship between a 100 ● ● ● ● classroom’s use of ● ● ● ● 90 ● ● the tutor & their ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● Average final math score ● ● ● average math score ● ● ● ● ● ● ● 80 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 70 ● ● ● ● ● ● • Current assumption ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● is that the slope of ● ● ● ● ● ● ● ● ● ● ● 60 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● this line (= tutor ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 50 ● ● ● ● effect) is the same ● 10 15 20 25 30 35 in every school Hours that a classroom used the tutor

  14. Assumption is that slope is Random Slopes the same for all schools • That is, so far, our model says that schools vary in baseline math score 70 70 70 • Random intercept • And that every 1 hour of 65 65 65 Average final math score tutor use ≈ 0.49 points Midland Crescent 60 60 60 gain in final math score • Slope is a fixed effect Highland 55 55 55 Different intercepts for each school 50 50 50 0 5 10 Hours that a classroom used the tutor

  15. Slopes now also differ Random Slopes across sampled schools • That is, so far, our model says that schools vary in baseline math score 1200 1200 1200 1200 70 70 70 Subject 2 • Random intercept 1100 1100 1100 1100 • And that every 1 hour of 65 65 65 1000 1000 1000 1000 Subject 4 Average final math score tutor use ≈ 0.49 points Midland 900 900 900 900 RT Crescent gain in final math score 60 60 60 Subject 3 800 800 800 800 Schools that start off Subject 1 • Slope is a fixed effect worse show a bigger 700 700 700 700 Highland • Schools may also vary 55 55 55 benefit of tutor 600 600 600 600 Schools still vary in in how much tutor affects intercept 500 500 500 500 50 50 50 their classrooms’ scores 1 1 1 1 2 2 2 2 3 3 3 3 0 5 10 • A random slope of tutor Word frequency Hours that a classroom used the tutor usage by schools • Such differences may correlate with baseline diff.s

  16. Random Slopes: Statistical Consequences • Important to capture this variability in the model • Observations from the same school will show more similar effects of tutor use • Different from what other schools would be like • If we don’t account Highland School Highland School for this similarity, (slope 0.82) (slope 0.82) non-independence Classroom 21, 27 hrs Classroom 22, 28 hrs that underestimates Highland School Highland School true variability (slope 0.82) (slope 0.82) Classroom 29, 25 hrs Classroom 30, 21 hrs Estimate t = Midland School Midland School Std. error (slope 0.17) (slope 0.17) Classroom 97, 27 hrs Classroom 98, 28 hrs • Inflates false positive rate

  17. Week 5: Random Slopes l Finish Nested Random Effects l Recap l Level-2 Variables l Multiple Random Effects l Random Slopes l Introduction l Notation l Implementation l Testing Random Effects l Crossed Random Effects l Examples l Random Slopes, Revisited l Between-Subjects & Within-Subjects Designs l Between-Items & Within-Items Designs l When are Random Slopes Necessary?

  18. Notation • • Level-2 model of classroom j : = + + β 0 jk γ 000 γ 100 x 1 j ( k ) U 0j(k) Tutor hours Overall baseline Classroom Teacher effect for this intercept classroom (Error) • Level 1 model of student i : = + Y i(jk) β 0 jk E i ( jk) End-of-year math Baseline for students in Student exam score this classroom Error

  19. Notation • Level-3 model of school k : = + δ 00k γ 000 V 00k School intercept School effect for this Overall baseline school (Error) • Level-2 model of classroom j : = + + β 0 jk δ 00k γ 100 x 1 j ( k ) U 0j(k) Tutor hours Baseline for classrooms Classroom Teacher effect for this in this school intercept classroom (Error) • Level 1 model of student i : = + Y i(jk) β 0 jk E i ( jk) End-of-year math Baseline for students in Student exam score this classroom Error

  20. Notation Right now, tutor slope γ 100 is still just a fixed value. Maybe, like the intercept, we should allow to vary across schools. • Level-3 model of school k : = + δ 00k γ 000 V 00k School intercept School effect for this Overall baseline school (Error) • Level-2 model of classroom j : = + + β 0 jk δ 00k γ 100 x 1 j ( k ) U 0j(k) Tutor hours Baseline for classrooms Classroom Teacher effect for this in this school intercept classroom (Error) • Level 1 model of student i : = + Y i(jk) β 0 jk E i ( jk) End-of-year math Baseline for students in Student exam score this classroom Error

  21. • Level-3 model of school k : = + δ 10k γ 100 V 10k Tutor slope By-school adjustment Overall tutor slope = for tutor slope (Error) + (fixed effect) δ 00k γ 000 V 00k School intercept School effect for this Overall baseline • Level-2 model of classroom j : school (Error) = + + β 0 jk δ 00k δ 10k x 1 j ( k ) U 0j(k) Tutor hours Baseline for classrooms Classroom Teacher effect for this in this school intercept classroom (Error) • Level 1 model of student i : = + Y i(jk) β 0 jk E i ( jk) End-of-year math Baseline for students in Student exam score this classroom Error

  22. • Level-3 model of school k : = + δ 10k γ 100 V 10k Tutor slope By-school adjustment Overall tutor slope = for tutor slope (Error) + (fixed effect) δ 00k γ 000 V 00k School intercept School effect for this Overall baseline • Level-2 model of classroom j : school (Error) = + + β 0 jk δ 00k δ 10k x 1 j ( k ) U 0j(k) Tutor hours Baseline for classrooms Classroom Teacher effect for this in this school intercept classroom (Error) • Level 1 model of student i : = + Y i(jk) β 0 jk E i ( jk) Let’s start doing our algebraic End-of-year math Baseline for students in Student substitution exam score this classroom Error

  23. • Level-3 model of school k : = + δ 10k γ 100 V 10k Tutor slope By-school adjustment Overall tutor slope = for tutor slope (Error) + (fixed effect) δ 00k γ 000 V 00k School intercept School effect for this Overall baseline • Level-2 model of classroom j : school (Error) = + + β 0 jk δ 00k δ 10k x 1 j ( k ) U 0j(k) Tutor hours Baseline for classrooms Classroom Teacher effect for this in this school intercept classroom (Error) • Level 1 model of student i : = + Y i(jk) β 0 jk E i ( jk) Let’s start doing our algebraic End-of-year math Baseline for students in Student substitution exam score this classroom Error

  24. • Level-3 model of school k : = + δ 10k γ 100 V 10k Tutor slope By-school adjustment Overall tutor slope for tutor slope (Error) (fixed effect) • Level-2 model of classroom j : = + + β 0 jk δ 10k x 1 j ( k ) γ 000 U 0j(k) Tutor hours + Overall baseline Classroom Teacher effect for this V 00k intercept classroom (Error) By-school adjustment for intercept (Error) • Level 1 model of student i : = + Y i(jk) β 0 jk E i ( jk) Let’s start doing our algebraic End-of-year math Baseline for students in Student substitution exam score this classroom Error

  25. • Level-3 model of school k : = + δ 10k γ 100 V 10k Tutor slope By-school adjustment Overall tutor slope for tutor slope (Error) (fixed effect) • Level-2 model of classroom j : = + + β 0 jk δ 10k x 1 j ( k ) γ 000 U 0j(k) Tutor hours + Overall baseline Classroom Teacher effect for this V 00k intercept classroom (Error) By-school adjustment for intercept (Error) • Level 1 model of student i : = + Y i(jk) β 0 jk E i ( jk) Let’s start doing our algebraic End-of-year math Baseline for students in Student substitution exam score this classroom Error

  26. • Level-2 model of classroom j : = + β 0 jk γ 000 (γ 100 + V 10k ) x 1 j ( k ) Tutor hours effect Overall baseline Classroom (overall & by school) intercept + + V 00k U 0j(k) By-school adjustment Teacher effect for this for intercept (Error) classroom (Error) • Level 1 model of student i : = + Y i(jk) β 0 jk E i ( jk) Let’s start doing our algebraic End-of-year math Baseline for students in Student substitution exam score this classroom Error

  27. • Level-2 model of classroom j : = + β 0 jk γ 000 (γ 100 + V 10k ) x 1 j ( k ) Tutor hours effect Overall baseline Classroom (overall & by school) intercept + + V 00k U 0j(k) By-school adjustment Teacher effect for this for intercept (Error) classroom (Error) • Level 1 model of student i : = + Y i(jk) β 0 jk E i ( jk) Let’s start doing our algebraic End-of-year math Baseline for students in Student substitution exam score this classroom Error

  28. • Mixed-effects model of student i : + = γ 000 (γ 100 + V 10k ) x 1 j ( k ) Y i(jk) Tutor hours effect Overall baseline End-of-year math (overall & by school) exam score + + V 00k U 0j(k) By-school adjustment Teacher effect for this for intercept (Error) classroom (Error) + E i ( jk) Student Error

  29. • Mixed-effects model of student i : Apply distributive property + = γ 000 (γ 100 + V 10k ) x 1 j ( k ) Y i(jk) Tutor hours effect Overall baseline End-of-year math (overall & by school) exam score + + V 00k U 0j(k) By-school adjustment Teacher effect for this for intercept (Error) classroom (Error) + E i ( jk) Student Error

  30. • Mixed-effects model of student i : + γ 100 x 1 j ( k ) = γ 000 Y i(jk) FIXED Fixed effect of tutor hours Overall baseline End-of-year math (overall) exam score + + + U 0j(k) V 10k x 1j(k) V 00k RANDOM By-school adjustment By-school adjustment Teacher effect for this for tutor slope (Error) for intercept (Error) classroom (Error) + E i ( jk) Student Error

  31. Covariance Matrix • Because we have more than one random variable, they can also covary [ The correlation Variance of subject ] parameter! intercept Covariance σ 2u00 j 0 cov( σ 2u00 j 0, u10 j 0 ) matrix for subject random cov( σ 2u00 j 0, u10 j 0 ) σ 2u10 j 0 effects in maximal model Variance of TutorHours slope across schools [ ] In near- σ 2u00 j 0 0 maximal model without the 0 σ 2u10 j 0 correlation parameter

  32. Week 5: Random Slopes l Finish Nested Random Effects l Recap l Level-2 Variables l Multiple Random Effects l Random Slopes l Introduction l Notation l Implementation l Testing Random Effects l Crossed Random Effects l Examples l Random Slopes, Revisited l Between-Subjects & Within-Subjects Designs l Between-Items & Within-Items Designs l When are Random Slopes Necessary?

  33. Random Slopes: Implementation + γ 100 x 1 j ( k ) = γ 000 Y i(jk) FIXED Fixed effect of tutor hours Overall baseline End-of-year math (overall) exam score + + + U 0j(k) V 10k x 1j(k) V 00k RANDOM By-school adjustment By-school adjustment Teacher effect for this for tutor slope (Error) for intercept (Error) classroom (Error) • So, lmer model: model.Slope <- lmer(FinalMathScore ~ • Next, we need overall (fixed) effects of Intercept and TutorHours

  34. Random Slopes: Implementation + γ 100 x 1 j ( k ) = γ 000 Y i(jk) FIXED Fixed effect of tutor hours Overall baseline End-of-year math (overall) exam score + + + U 0j(k) V 10k x 1j(k) V 00k RANDOM By-school adjustment By-school adjustment Teacher effect for this for tutor slope (Error) for intercept (Error) classroom (Error) • So, lmer model: model.Slope <- lmer(FinalMathScore ~ 1 + • TutorHours Next, we need overall (fixed) effects of Intercept and TutorHours

  35. Random Slopes: Implementation + γ 100 x 1 j ( k ) = γ 000 Y i(jk) FIXED Fixed effect of tutor hours Overall baseline End-of-year math (overall) exam score + + + U 0j(k) V 10k x 1j(k) V 00k RANDOM By-school adjustment By-school adjustment Teacher effect for this for tutor slope (Error) for intercept (Error) classroom (Error) • So, lmer model: model.Slope <- lmer(FinalMathScore ~ 1 + • TutorHours How about a random difference in the intercept for each ClassroomID?

  36. Random Slopes: Implementation + γ 100 x 1 j ( k ) = γ 000 Y i(jk) FIXED Fixed effect of tutor hours Overall baseline End-of-year math (overall) exam score + + + U 0j(k) V 10k x 1j(k) V 00k RANDOM By-school adjustment By-school adjustment Teacher effect for this for tutor slope (Error) for intercept (Error) classroom (Error) • So, lmer model: model.Slope <- lmer(FinalMathScore ~ 1 + • TutorHours + (1|ClassroomID) How about a random difference in the intercept for each ClassroomID?

  37. Random Slopes: Implementation + γ 100 x 1 j ( k ) = γ 000 Y i(jk) FIXED Fixed effect of tutor hours Overall baseline End-of-year math (overall) exam score + + + U 0j(k) V 10k x 1j(k) V 00k RANDOM By-school adjustment By-school adjustment Teacher effect for this for tutor slope (Error) for intercept (Error) classroom (Error) • So, lmer model: model.Slope <- lmer(FinalMathScore ~ 1 + • TutorHours + (1|ClassroomID) Lastly, both intercept and TutorHours slope need to vary across schools

  38. Random Slopes: Implementation + γ 100 x 1 j ( k ) = γ 000 Y i(jk) FIXED Fixed effect of tutor hours Overall baseline End-of-year math (overall) exam score + + + U 0j(k) V 10k x 1j(k) V 00k RANDOM By-school adjustment By-school adjustment Teacher effect for this for tutor slope (Error) for intercept (Error) classroom (Error) • So, lmer model: model.Slope <- lmer(FinalMathScore ~ 1 + • TutorHours + (1|ClassroomID) + (1+TutorHours|School) Lastly, both intercept and TutorHours slope need to vary across schools

  39. Random Slopes: Implementation + γ 100 x 1 j ( k ) = γ 000 Y i(jk) FIXED Fixed effect of tutor hours Overall baseline End-of-year math (overall) exam score + + + U 0j(k) V 10k x 1j(k) V 00k RANDOM By-school adjustment By-school adjustment Teacher effect for this for tutor slope (Error) for intercept (Error) classroom (Error) • So, lmer model: model.Slope <- lmer(FinalMathScore ~ 1 + • TutorHours + (1|ClassroomID) + (1+TutorHours|School), data=tutor)

  40. Random Slopes: Implementation • Here’s our final model again: model.Slope <- lmer(FinalMathScore ~ • 1 + TutorHours + (1|ClassroomID) + (1 + TutorHours|School), data=tutor) Like a miniature model formula for things we think will vary by schools Schools differ in their Schools differ in the intercept (baseline math effectiveness of the tutor score) on their math scores

  41. Random Slopes: Output NEW: How much schools vary from the mean Correlation: Schools TutorHours slope with a lower starting score show a larger benefit of the tutor Still have a fixed effect of TutorHours— slope estimated across all subjects & classrooms The fixed effect is reliable. In this case, we can conclude that final math scores are generally higher for classrooms that used the tutor more, even if there are some differences across schools in the size of this effect.

  42. Random Slopes: Model Comparison As compared to the intercept-only model, our estimate of baseline school variance has now decreased—many of these unexplained “baseline” differences have now been explained as differing effectiveness of the tutor OLD MODEL (INTERCEPT ONLY) NEW MODEL (WITH SLOPE) • Now that we’ve properly accounted for clustering, our estimate of the standard error of the tutor effect is higher … and t value lower • Excluding the slope would have increased our Type I error rate

  43. Random Effects: Implementation • By the way, how come we never discussed a (1+TutorHours|Classroom) slope? • Within a school , we Class 28 can calculate a 85 Class 25 ● ● regression line Class 24 ● Class 27 80 Average final math score ● Class 21 relating tutor use to ● Class 23 ● Class 29 ● Class 22 average math score ● 75 Class 26 ● • Each class = 1 point 70 Class 30 ● 65 20 22 24 26 28 30 32 Hours that a classroom used the tutor

  44. Random Effects: Implementation • By the way, how come we never discussed a (1+TutorHours|Classroom) slope? • But each class has only a single value of 85 TutorHours 80 • Classroom 22 used it Average final math score for 28 hours Class 22 ● 75 • No way to draw a line relating different values 70 of TutorHours to score 65 • Need 2 points for a line! 20 22 24 26 28 30 32 Hours that a classroom used the tutor

  45. Week 5: Random Slopes l Finish Nested Random Effects l Recap l Level-2 Variables l Multiple Random Effects l Random Slopes l Introduction l Notation l Implementation l Testing Random Effects l Crossed Random Effects l Examples l Random Slopes, Revisited l Between-Subjects & Within-Subjects Designs l Between-Items & Within-Items Designs l When are Random Slopes Necessary?

  46. Testing Random Effects • Does this random slope contribute significantly? • i.e., is there significant variation in the effectiveness of the tutor across schools? • We can compare the fit of models with & without the random slope • Using the likelihood-ratio test • Same as when we fixed effects last week

  47. Testing Random Effects • Does this random slope contribute significantly? • i.e., is there significant variation in the effectiveness of the tutor across schools? • We can compare the fit of models with & without the random slope • Using the likelihood-ratio test • Same as when we tested fixed effects last week anova(model.Intercepts, model.Slope) • • Could test random intercepts the same way Model with slope fits significantly better!

  48. Testing Random Effects • Remember that this tests whether there is significant variation in the slope across schools • i.e., more than expected under H 0 of no variation • More complex model will always fit numerically the same or better • But if three schools had slopes of 0.39, 0.38, and 0.40, this would probably not be significant • Consistent with what’s expected from sampling error

  49. Testing Random Effects • Caveats: • Overfitting / shrinkage • Problem we talked about last week about using the data to decide the model. “Using up” degrees of freedom • A new sample might yield a different random effects structure • In cases where we have a clear sampling design, would want model to reflect that • e.g., weird not to include random intercepts of classroom & school here

  50. Week 5: Random Slopes l Finish Nested Random Effects l Recap l Level-2 Variables l Multiple Random Effects l Random Slopes l Introduction l Notation l Implementation l Testing Random Effects l Crossed Random Effects l Examples l Random Slopes, Revisited l Between-Subjects & Within-Subjects Designs l Between-Items & Within-Items Designs l When are Random Slopes Necessary?

  51. An Experimental Dataset • naming.csv • RT to name/read a word aloud for students learning English • 60 Subjects each presented with 49 words (Items) we randomly picked out of a dictionary • Each row is a single trial (one subject responding to one word) • We’re interested in YearsOfStudy (of English) and WordFreq (word frequency) • Also interested in their interaction • Which of these should we consider Fixed Effects? Which are Random Effects?

  52. An Experimental Dataset • naming.csv • RT to name/read a word aloud for students learning English • 60 Subjects each presented with 49 words (Items) we randomly picked out of a dictionary • Each row is a single trial (one subject responding to one word) • We’re interested in YearsOfStudy (of English) and WordFreq (word frequency) • Also interested in their interaction • Which of these should we consider Fixed Effects? Which are Random Effects? • Fixed: YearsOfStudy, WordFreq, & interaction • Random: Subject, Item

  53. Crossed Random Effects Subject Subject Subjects (Level-2) 1 2 RT 1 RT 2 RT 3 RT 4 Trials (Level-1) • Now, we have >1 trial per Subject • Level-1 observations (RTs) are nested within subjects • Clear we need to take account of this • Some subjects will have faster RTs in general than others

  54. Crossed Random Effects • Try an R command to look at the categories of the Item variable. Do some seem easier than others?

  55. Crossed Random Effects • Try an R command to look at the categories of the Item variable. Do some seem easier than others? summary(naming$Item) • Probably relatively Probably relatively easy hard!

  56. Crossed Random Effects Subject Subject Subjects (Level-2) 1 2 RT 1 RT 2 RT 3 RT 4 Trials (Level-1) “Boy” “Carbu Words retor” • Each word is also used in more than 1 trial • Want to take account of that, too • Observations from the same item will be more similar to one another, too (“boy” easier than “carburetor”)

  57. Crossed Random Effects Subject Subject Subjects (Level-2) 1 2 RT 1 RT 2 RT 3 RT 4 Trials (Level-1) “Boy” “Carbu Words (Level-2) retor” • In fact, we can think of each trial as the pairing of a subject and a word

  58. Crossed Random Effects Subject Subject Subjects (Level-2) 1 2 RT 1 RT 2 RT 3 RT 4 Trials (Level-1) “Boy” “Carbu Words (Level-2) retor” • Random effects not hierarchically nested • Before: Each classroom appears in only 1 school • Here: Each item presented to each subject • If we draw all the lines in, we see they cross

  59. Crossed Random Effects Subject Subject Subjects (Level-2) 1 2 RT 1 RT 2 RT 3 RT 4 Trials (Level-1) “Boy” “Carbu Words (Level-2) retor” • Crossed random effects structure • a/k/a cross-classified when we’re dealing with existing classifications

  60. Crossed Random Effects Subject Subject Subjects (Level-2) 1 2 RT 1 RT 2 RT 3 RT 4 Trials (Level-1) “Boy” “Carbu Words (Level-2) retor” • Conceptually different sampling, but same syntax! • Try fitting a model with: • Fixed effects of YearsOfStudy , WordFreq , & interaction • Random intercepts for Subject and Item

  61. Crossed Random Effects model1 <- lmer(RT ~ 1+ YearsOfStudy * WordFreq + • (1|Subject) + (1|Item), data=naming) Greater Subject variance than Item variance Typical in experiments because we often design items to be similar 2 significant main effects, but no interaction

  62. Crossed Random Effects Subject Subject Subjects (Level-2) 1 2 RT 1 RT 2 RT 3 RT 4 Trials (Level-1) “Boy” “Carbu Words (Level-2) retor” • Huge improvement over ANOVA analyses in psycholinguistics / experimental psychology! • We used to have to conduct separate analyses to assess generalizability over subjects & items • Or, item effects were simply ignored!

  63. Week 5: Random Slopes l Finish Nested Random Effects l Recap l Level-2 Variables l Multiple Random Effects l Random Slopes l Introduction l Notation l Implementation l Testing Random Effects l Crossed Random Effects l Examples l Random Slopes, Revisited l Between-Subjects & Within-Subjects Designs l Between-Items & Within-Items Designs l When are Random Slopes Necessary?

  64. Between vs Within Subjects • What kind of random slopes might be relevant here? • To answer this question, we need to understand how “between-subjects variables” differ from “within-subjects variables”

  65. Between vs Within Subjects • In an experimental design, some variables are: Ø Between-Subjects variables : Each subject is in 1 and only 1 group … or has 1 and only 1 value • Randomly assigned to drug 1 vs. drug 2 vs. placebo • Demographic variables; e.g., SES • Cognitive/linguistic differences (e.g., working mem. score) • “Between subjects” because differences in this variable are only seen between one subject and another SUBJECT 10’s SUBJECT 11’s DATA DATA Native speaker Non-native speaker Trial 1: Correct Trial 1: Incorrect Trial 2: Correct Trial 2: Correct Trial 3: Incorrect Trial 3: Incorrect

  66. Between vs Within Subjects • In an experimental design, some variables are: Ø Between-Subjects variables : Each subject is in 1 and only 1 group … or has 1 and only 1 value • Randomly assigned to drug 1 vs. drug 2 vs. placebo • Demographic variables; e.g., SES • Cognitive/linguistic differences (e.g., working mem. score) • “Between subjects” because differences in this variable are only seen between one subject and another Ø Within-Subjects variables: Same subject sees more than 1 condition or has >1 value • Same subject sees both congruent (green) and incongruent (blue) Stroop trials • Values that vary w/in a study, e.g., # of previous trials • Variables where you’d use a repeated measures ANOVA • “Within-subjects” because you can see differences in this variable even within a single subject

  67. Between vs Within Subjects • In an experimental design, some variables are: Ø Between-Subjects variables : Each subject is in 1 SUBJECT 12’S DATA and only 1 group … or has 1 and only 1 value • Randomly assigned to @ Trial 1: Congruent Stroop, 655 ms Trial 2: Incongruent Stroop, 512 ms • Demographic variables; e.g., SES Trial 3: Incongruent Stroop, 711 ms • Cognitive/linguistic differences (e.g., working mem. score) Trial 4: Congruent Stroop: 642 ms • “Between subjects” because differences in this variable are only seen between one subject and another Ø Within-Subjects variables: Same subject sees more than 1 condition or has >1 value • Same subject sees both congruent (green) and incongruent (blue) Stroop trials • Values that vary w/in a study, e.g., # of previous trials • Variables where you’d use a repeated measures ANOVA • “Within-subjects” because you can see differences in this variable even within a single subject

  68. Between vs Within: Advanced • The same variable could end up as between- or within-subjects, depending on experimental design • I’m interested in maintenance rehearsal (repetition) vs. elaborative rehearsal (relating to other concepts) • Half of my participants study second-language vocab using maintenance rehearsal, and half study words using elaborative rehearsal à Between Subjects • Each participant studies some words with maintenance rehearsal and some with elaborative rehearsal à Within Subjects

  69. Between vs Within: Practice • I assign 40 students to study for an upcoming biology exam by practicing retrieving the facts ( retrieval practice ) and 40 students to re-read the textbook ( restudy ) . Practice Type is… • In a visual perception task, 20 subjects try to determine as quickly as possible whether a circle is present on the screen (among other objects). Each subject sees ½ of the trials where the square is present and ½ where it’s not. Trial Type is… • Kevin compares 100 first-generation college students to 100 college students who are not first- generation students in their feeling of belonging at their university. College history is…

  70. Between vs Within: Practice • I assign 40 students to study for an upcoming biology exam by practicing retrieving the facts ( retrieval practice ) and 40 students to re-read the textbook ( restudy ) . Practice Type is… • Between subjects • In a visual perception task, 20 subjects try to determine as quickly as possible whether a circle is present on the screen (among other objects). Each subject sees ½ of the trials where the square is present and ½ where it’s not. Trial Type is… • Within subjects • Kevin compares 100 first-generation college students to 100 college students who are not first- generation students in their feeling of belonging at their university. College history is… • Between subjects

  71. Between vs Within: More Practice • I recruit 40 Psych Subject Pool students for my experiment. Each participant reads 20 syntactically ambiguous sentences and 20 unambiguous sentences. Sentence type is… • I recruit 40 Psych Subject Pool students for my experiment. 20 participants read syntactically ambiguous sentences, and the other 20 read only unambiguous sentences. Sentence type is…

  72. Between vs Within: More Practice • I recruit 40 Psych Subject Pool students for my experiment. Each participant reads 20 syntactically ambiguous sentences and 20 unambiguous sentences. Sentence type is… • Within subjects • I recruit 40 Psych Subject Pool students for my experiment. 20 participants read syntactically ambiguous sentences, and the other 20 read only unambiguous sentences. Sentence type is… • Between subjects

  73. Between vs Within: More Practice • Each of 100 participants in my study makes a decision in each of 2 different moral dilemmas: one involving direct harm, and one involving indirect harm. I also measure participants’ working memory and split them into “low WM” and “high WM” groups. • Moral dilemma type is… • Working memory is…

  74. Between vs Within: More Practice • Each of 100 participants in my study makes a decision in each of 2 different moral dilemmas: one involving direct harm, and one involving indirect harm. I also measure participants’ working memory and split them into “low WM” and “high WM” groups. • Moral dilemma type is… • Within subjects • Working memory is… • Between subjects

  75. Between vs Within: More Practice • How about in our naming.csv dataset? • Word frequency is… • Each subject sees high-, medium- and low-frequency words • Within subjects • Years of study is… • This experiment takes place at a single point in time, so the number of years a subject has been studying English is fixed and never varies • Between subjects

  76. When are Random Slopes Appropriate? • If the random effect is subject… • Random slopes for subjects appropriate for: • Within-subjects variables • We can draw a regression line for each subject • Calculate the effect “within” each subject Subject 1 High Frequency Subject 1 Low Frequency Subject 2 High Frequency Subject 2 Low Frequency • Random slopes for subjects inappropriate for: • Between-subjects variables • Can’t draw a regression line within each subject Subject 1: 6 years of study Subject 2: 2 years of study

  77. Random Slopes: Implementation • Remember, original model was: model1 <- lmer(RT ~ 1 + WordFreq * YearsOfStudy • + (1|Subject) + (1|Item), data=naming) • What variable varies within subjects? Try adding a random slope for it

  78. Random Slopes: Implementation • Remember, original model was: model1 <- lmer(RT ~ 1 + WordFreq * YearsOfStudy • + (1|Subject) + (1|Item), data=naming) • With slope: model2 <- lmer(RT ~ 1 + WordFreq * YearsOfStudy • + (1+WordFreq|Subject) + (1|Item), data=naming) Again, miniature model formula for things we think will vary by subjects Subjects differ in their Subjects differ in the intercept (baseline RT) effect of word frequency on their RTs

  79. Slope is the same for Random Slopes everyone • Original model says that subjects vary in baseline RT 1200 1200 1200 1200 Subject 2 • Random intercept 1100 1100 1100 1100 • And that a 1-unit change 1000 1000 1000 1000 Subject 4 in word frequency ≈ 120 900 900 900 900 RT ms decrease in RT Subject 3 800 800 800 800 Subject 1 • Slope is a fixed effect 700 700 700 700 600 600 600 600 Different intercepts for each subject 500 500 500 500 1 1 1 1 2 2 2 2 3 3 3 3 Word frequency

Recommend

Business and Business Environment Business and Business Environment Introduction Business is

Business and Business Environment Business and Business Environment Introduction Business is

Business and Business Environment Business and Business Environment Introduction Business is the organised efforts and activities of individuals to produce and sell goods and services for profit. Business commence with a business concept

550 views • 24 slides
Highlights Business Updates Industry Trends Highlights Business Updates Industry Trends Key

Highlights Business Updates Industry Trends Highlights Business Updates Industry Trends Key

Capital Asset &amp; M otilal O swal Market Wealth Business Business F inancial S ervices Housing Fund Earnings Presentation | Q3FY17 Finance Based Business Business M arching O nwards with F ocused S trategies Highlights Business

979 views • 32 slides
Business Presentations Business Presentations Business presentations can be to inform

Business Presentations Business Presentations Business presentations can be to inform

Business Presentations Business Presentations Business presentations can be to inform to promote to sell to solve problems Business Presentations How is a business presentation different to a public speech?

644 views • 23 slides
Doing Business in the Defence Sector Who we are Nova Scotia Business Inc. (NSBI) Business

Doing Business in the Defence Sector Who we are Nova Scotia Business Inc. (NSBI) Business

Doing Business in the Defence Sector Who we are Nova Scotia Business Inc. (NSBI) Business Development Agency Making connections Attracting new companies Growing exports Your first point of contact in realizing business

1.34k views • 95 slides
Business CorrespondenceRoutine business transactions!

Business CorrespondenceRoutine business transactions!

Homework Videos Business CorrespondenceRoutine business transactions! https://github.com/drbean/curriculum/trunk/correspondence Mon 09 Dec 2019 02:29:08 PM CST Business CorrespondenceRoutine business transactions! Dr Bean ( )

472 views • 7 slides
Business Interactions w/Intergovernmental Organizations Business and business NGOs are a

Business Interactions w/Intergovernmental Organizations Business and business NGOs are a

Business Interactions w/Intergovernmental Organizations Business and business NGOs are a recognized major group (Agenda21) (one of several: trade unions, local authorities, youth, women, &amp;c...) International Organisation of

221 views • 4 slides
BUSINESS WRITING NEAR EAST UNIVERSITY 1 2 Business Writing Business Writing Follow these

BUSINESS WRITING NEAR EAST UNIVERSITY 1 2 Business Writing Business Writing Follow these

BUSINESS WRITING NEAR EAST UNIVERSITY 1 2 Business Writing Business Writing Follow these rules Topics to cover Business letters State your purpose Envelopes Be straightforward, clear, concise, objective and courteous Job

449 views • 8 slides
Business Plan The Business Definition The offer what will the business offer the

Business Plan The Business Definition The offer what will the business offer the

Business Plan The Business Definition The offer what will the business offer the customers? Target market who will it serve? Production and delivery capability how will it provide the products and services it sells? Types of

1.09k views • 40 slides
Business Plan The Business Definition The offer what will the business offer the

Business Plan The Business Definition The offer what will the business offer the

Business Plan The Business Definition The offer what will the business offer the customers? Target market who will it serve? Production and delivery capability how will it provide the products and services it sells? Types of

914 views • 39 slides
Business Letters Mr. Gordon Why do we need to write business letters? The key to

Business Letters Mr. Gordon Why do we need to write business letters? The key to

Business Letters Mr. Gordon Why do we need to write business letters? The key to communication Business to Business Individual to Business Business to Individual Accomplish a goal Earn a business respect Get what we

631 views • 32 slides
The Business of Bus Busin ines ess s is is Business Milton Friedman The Business of

The Business of Bus Busin ines ess s is is Business Milton Friedman The Business of

The Business of Bus Busin ines ess s is is Business Milton Friedman The Business of Business They did not teach you how to run a business in law school or medical school Two foundational concepts for success:

486 views • 15 slides
Build a Profitable Business Workshop Tell us about you and your business 1) Your name

Build a Profitable Business Workshop Tell us about you and your business 1) Your name

inspire, educate, equip entrepreneurs to build profitable businesses Build a Profitable Business Workshop Tell us about you and your business 1) Your name 2)Business name 3)Business stage (idea, early, growth, late) 4)What product (do/will)

571 views • 23 slides
Asian Seafoods Coldstorage PLC. Q2 Opportunity Day 08 th Aug 18 TUNA BUSINESS FEED BUSINESS

Asian Seafoods Coldstorage PLC. Q2 Opportunity Day 08 th Aug 18 TUNA BUSINESS FEED BUSINESS

Asian Seafoods Coldstorage PLC. Q2 Opportunity Day 08 th Aug 18 TUNA BUSINESS FEED BUSINESS FROZEN BUSINESS PETFOOD BUSINESS DISTRIBUTION BUSINESS We are the right choice! Disclaimer This presentation includes forward-looking statements

549 views • 43 slides
Business Update for 2018 3 Financial Highlight 4 Q&amp;A 5 1 Business Overview ADB Business

Business Update for 2018 3 Financial Highlight 4 Q&amp;A 5 1 Business Overview ADB Business

Applied DB Public Company Limited ADB Opportunity Day March 28, 2018 Agenda Business Overview 1 Market Trend and Business Opportunity 2 Business Update for 2018 3 Financial Highlight 4 Q&amp;A 5 1 Business Overview ADB Business

400 views • 39 slides
WE R THE RIGHT CHOICE Frozen Tuna Pet Food Feed Distribution Business Business Business

WE R THE RIGHT CHOICE Frozen Tuna Pet Food Feed Distribution Business Business Business

WE R THE RIGHT CHOICE Frozen Tuna Pet Food Feed Distribution Business Business Business Business Business Asian Seafoods Coldstorage PLC. . Corporate Access - 24

587 views • 30 slides
College of Arts, Sciences, and Business Transfer Conference Fall 2016 Business 200+

College of Arts, Sciences, and Business Transfer Conference Fall 2016 Business 200+

College of Arts, Sciences, and Business Transfer Conference Fall 2016 Business 200+ employers recruit S&amp;T business majors Business degrees are AACSB accredited Average Salaries: Business &amp; Management Systems $54,750

111 views • 9 slides
Grow your business by better financial management and accounting for SME 4 October 2016 GNZ

Grow your business by better financial management and accounting for SME 4 October 2016 GNZ

Grow your business by better financial management and accounting for SME 4 October 2016 GNZ Business Alliances BUSINESS IS A SEPARATE ENTITY Owners Others BUSINESS Directors Mgmt/ Staff GNZ Business Alliances BUSINESS CYCLE Customer

517 views • 14 slides
Business Office Site Business Office Business Office OUR SERVICES (1 of 2) Business Office OUR

Business Office Site Business Office Business Office OUR SERVICES (1 of 2) Business Office OUR

Business Office Site Business Office Business Office OUR SERVICES (1 of 2) Business Office OUR SERVICES (2 of 2) Business Office A. DEPOSITS &amp; BILLING 1. Incoming ACH &amp; Wires 2. Check Deposit 3. Third Party Billing 4. How

801 views • 16 slides
Finance and business: The numbers you need to know 1 Our Purpose at Vision Beyond Business is

Finance and business: The numbers you need to know 1 Our Purpose at Vision Beyond Business is

03/09/2019 Finance and business: The numbers you need to know 1 Our Purpose at Vision Beyond Business is to help business owners and organisations succeed and to create a positive impact for small businesses, our team, our community and

314 views • 9 slides
Getting Your Business Idea Off the Ground! Speaker Martha Londagin, Executive Business

Getting Your Business Idea Off the Ground! Speaker Martha Londagin, Executive Business

Getting Your Business Idea Off the Ground! Speaker Martha Londagin, Executive Business Consultant Former business attorney and commercial banker, current small business owner for 16 years Visit startupjunkie.org to see all our

559 views • 28 slides
Asian Seafoods Coldstorage PLC. Opportunity Day-14 th June 2017 TUNA BUSINESS FEED BUSINESS

Asian Seafoods Coldstorage PLC. Opportunity Day-14 th June 2017 TUNA BUSINESS FEED BUSINESS

Asian Seafoods Coldstorage PLC. Opportunity Day-14 th June 2017 TUNA BUSINESS FEED BUSINESS FROZEN BUSINESS PETFOOD BUSINESS DISTRIBUTION BUSINESS We are the right choice! Disclaimer This presentation includes forward-looking statements and

1k views • 55 slides
Robit Plc 2 Agenda Our Business Business Structure Business Segment Top Hammer Business

Robit Plc 2 Agenda Our Business Business Structure Business Segment Top Hammer Business

Robit Plc Growth Company Robit Plc Growth Company Financial Statements Review 2015 Robit Plc 2 Agenda Our Business Business Structure Business Segment Top Hammer Business Segment Down-the-Hole Business Segment Digital

365 views • 19 slides
Intelligent Solutions for your Intelligent Solutions for your Business Challenges Business

Intelligent Solutions for your Intelligent Solutions for your Business Challenges Business

Intelligent Solutions for your Intelligent Solutions for your Business Challenges Business Challenges www.silvertouch.com SAP Business One Agenda SAP Business One Features &amp; Success Story SAP Business One Basic Modules Do More

550 views • 33 slides
Business Growth Support Event Boston July 2019 Business Growth: UKs support for business

Business Growth Support Event Boston July 2019 Business Growth: UKs support for business

Business Growth Support Event Boston July 2019 Business Growth: UKs support for business in particular SMEs Our modern Industrial Strategy aims to make the UK the best place to start and grow a business. Within this, our focus is on:

972 views • 64 slides

More recommend