Does Mode Matter For Modeling Political Choice? Evidence From the - PowerPoint PPT Presentation

Does Mode Matter For Modeling Political Choice? Evidence From the 2005 British Election Study by Harold Clarke David Sanders Marianne Stewart Paul Whiteley

Survey Modes For National Election Studies - Pros & Cons • Traditional In-Person - VERY Expensive (Big Chunks of NSF & ESRC PSCI Budgets!), Very Slow • RDD Telephone - Increasingly Expensive, Fast • Self-Completion Mail Questionnaires - Inexpensive, Slow • Internet - Inexpensive, VERY Fast

The Rap on Internet Surveys: Limited Coverage and NonProbability Samples • In-Person Surveys - The ANES Gold Standard • RDD Surveys - The CES Gold Standard • In-Person & RDD Surveys - Probability Samples, but Potential Respondents Select Out • Unit Non-Response - Now Large in both In-Person and RDD, Sometimes Huge in RDD • Internet Surveys - Non-Probability Samples (but KN), Potential Respondents Select In • All Modes Have Selection Biases

The 2005 BES

Figure 1: Probability and Internet Panel Survey Design in the 2005 British Election Study BES 2005 CORE FACE-TO-FACE PANEL Wave 1 Pre-election Wave 2 Post-election Wave 3 One Year Out Probability Sample, Probability Sample, Internet users from Wave Face-to-Face N=3589 Face-to-Face N=4161 2 Probability Sample, Internet 128 PSUs Including top-up, Survey Method N=983 mail-back; 128 PSUs Probability Internet sample versus traditional Face-to-face vs Internet sample Face-to-face vs Internet sampling Sampling Internet sampling experiment (1) Experiment: experiment (2) BES 2005 INTERNET CAMPAIGN PANEL SURVEY: Wave 4 Wave 1 Wave 2 Wave 3 One Year Out Campaign survey Post-election Pre-campaign Interview 200 interviews per Interview Baseline Survey N=6186 Day for 30 days N=5910 N=7793 N=6068

Survey Houses • In-Person -> National Center for Social Research ‘Natcen’ - conducted 1983 - 1997 BES • Internet -> YouGov - also conducting NSF-sponsored 'Valence Politics and the Dynamics of Party Support' Project

Figure 2. Reported Vote In-Person and Internet Post-Election Surveys and Actual Vote in Britain, 2005 General Election 50 45 39.6 40 36.2 36.1 33.2 35 31.7 30.8 30 Percentage 24.4 25 22.6 22.4 20 15 8.7 7.9 10 6.3 5 0 Labour Conservative Liberal Democrat Other Parties In-Person Internet Actual Vote

Figure 3. Party Identification in Pre- and Post- Election In-Person and Internet Surveys 50 45 40 37 36 34 33 35 Percentage 30 26 25 25 24 23 23 25 21 18 20 13 15 12 12 11 9 10 7 6 6 5 0 Labour Conservative Liberal Other Parties None, DK Democrat In-Person Pre Internet Pre In-Person Post Internet Post

Figure 4. Reported Turnout in In-Person and Internet Surveys and Actual Turnout in 2005 British General Election 100 90 82.9 80 71.7 70 61.1 60 Percentage 50 40 30 20 10 0 In-Person Internet Actual

Data Quality? Comparative Overeports of Turnout in National Election Studies 2005 BES, internet 21.8 2005 BES, IP 10.6 1964-2001 BES, IP 9.9 2001 BES, RDD 18.7 2000 CES, RDD 21.5 2004 CES, RDD 25.2 2000 ANES, RDD, rev 27.2 2000 ANES, IP, rev 20.2 2002 ANES, RDD, rev 17.8 2002 ANES, RDD, trad 37.9 2004 ANES, IP, rev 17.3 2004 ANES, IP, trad 24.7 2000 NAES, RDD 24.8 2004 NAES, RDD 34.7 0 5 10 15 20 25 30 35 40

Composite Labour Vote Model • Party Leader Images • Party Best Most Important Issue • Party Identification • Party-Issue Proximities • Economic Evaluations • Opinions about Iraq War • Tactical Voting • Demographics

Table 5. Comparative Performance of Rival Party Choice Models McFadden R2 McKelvey R2 AIC BIC A. Models Estimated Using In-Person Survey Data Social Class .01 .02 2794.20 2805.51 All Demographics .03. .06 2753.54 2810.08 Economic Evaluations .07 .13 2633.38 2644.69 Issue Proximities .12 .22 2507.63 2530.25 Most Important Issue .27 .40 2079.75 2108.02 Party Identification .37 .48 1794.87 1823.14 Leader Images .40 .65 1692.95 1715.56 Composite Model .58 .76 1256.45 1414.76 B. Models Estimated Using Internet Survey Data Social Class .01 .01 6409.16 6422.16 All Demographics .02 .04 6328.65 6400.17 Economic Evaluations .14 .24 5564.96 5577.97 Issue Proximities .19 .34 5229.46 5255.52 Most Important Issue .33 .48 4299.71 4332.29 Party Identification .36 .50 4163.88 4196.45 Leader Images .44 .64 3617.93 3643.94 Composite Model .59 .76 2715.98 2898.40

Table 6. Rival Models of Labour Voting Comparative Predictive Power In-Person Survey In ternet Survey % Correctly % Correctly Predicted Lambda Predicted Lambda Models Social Class 60.5 .00 63.9 .00 All Demographics 63.1 .07 64.3 .01 Economic Evaluations 65.3 .12 70.4 .18 Issue Proximities 67.9 .19 72.3 .23 Most Important Issue 78.3 .45 80.8 .47 Party Identification 83.5 .59 82.6 .52 Leader Images 82.0 .55 83.7 .55 Composite Model 87.3 .68 88.6 .68

Figure 5. Cross-Predicting Labour Voting in the In-Person and Internet Surveys 100 88.6 88.1 87.3 86.8 90 80 Percent Correctly Classified 70 60 50 40 30 20 10 0 In-Person Data Internet Data In-Person Model Internet Model

Conclusions • Mode Doesn’t Matter for Modeling Electoral Choice in Britain • Internet Surveys – The Future? • Very Cost Effective • Huge N’s - Study Election Outcomes • Super Fast • Cool Experiments – e.g., Feedback to Respondents • Do British Findings Travel Well? How far is it from Wivenhoe Park to Ann Arbor? To Montreal? Encouraging Findings from our 2006 Congressional & 2006 Canadian election studies

The 2009/10 BES • More Mode Comparisons • Survey Experiments • Huge Internet Campaign Survey • Monthly Continuous Monitoring Survey, with Research Opportunities Like TESS – you can send us your proposal! • Links to CCAP, and hopefully ANES and PSNZ