External Validity March 25 1 / 16 Definition How do we define - PowerPoint PPT Presentation

External Validity March 25 1 / 16

Definition How do we define external validity? Mundane realism (surface similarity) Generalizability of results 2 / 16

Discussion Experiments have strong internal validity. If that is why we do experiments, why do we care about the external validity of experiments? 3 / 16

SUTO framework Setting Units Treatments Outcome 4 / 16

Assessing external validity How do we assess the external validity of a single study? 5 / 16

Assessing external validity The most common multi-study method of assessing external validity is the review 1. Qualitative reviews 2. Quantitative/systematic reviews (meta-analysis) 6 / 16

Assessing external validity Other strategies of assessing external validity include: Formal/exact replication Approximate replication Statistical generalization Parallel experimentation 7 / 16

Exact replication Recreate a previous experiment exactly Requires a complete protocol SUTO: What can we replicate? 8 / 16

Approximate replication Model a new study off a previous study Possibly intentional changes in SUTO characteristics 9 / 16

Statistical Generalization Hedges is interested in the formal, statistical generalization of experiments How does this work? 10 / 16

Parallel Experiments Execute the same experiment with variations in some experimental feature Compare results across experiments SUTO: What can we replicate? 11 / 16

Discussion Are these types of replication useful? Why or why not? 12 / 16

Discussion How important is it for an experiment to have high external validity? 13 / 16

Discussion What is the point of a single experiment if it does not generalize? 14 / 16

Presentations What are you thinking for your exam paper? 15 / 16

Preview No class next week In 2 weeks: Survey Experiments Presentation by Rune Slothuus Share a one-page synopsis of your project Due via email to me: Monday April 7 12:00 16 / 16