Interviewer Effects in Real and Falsified Interviews: Results from a Large Scale Experiment Conference “New Techniques and Technologies for Statistics” (NTTS) Brussels, 10-12/03/2015 Uta Landrock, Peter Winker, Karl-Wilhelm Kruse, Natalja Menold
Overview • Introduction: o Project background o Falsifications and interviewer effects • Procedure: o Data base o Method • Results: Interviewer effects • Conclusions
Introduction: Project background • DFG (German Research Foundation) funded research project “ Identification of Falsifications in Survey Data ” • Teams: o Justus-Liebig-University Giessen , Germany, (Prof. P. Winker; K. W. Kruse) GESIS – Leibniz Institute for the Social Sciences , o Germany, (Dr. N. Menold; U. Landrock)
Introduction: Falsifications and interviewer effects “ Interviewer falsification means the intentional departure from the designed interviewer guidelines or instructions, unreported by the interviewer, which could result in the contamination of data.” AAPOR (2003) • Falsifications in survey data affect all further steps of analysis (Schräpler and Wagner, 2003) • We analyse influences of interviewers’ characteristics and payment schemes on real and falsified data.
Procedure: Data base • Interviewers conducted real face-to-face interviews (N=710). • The same interviewers received basic socio- demographic information about real survey participants interviewed by his/her colleagues; the interviewers were instructed to falsify corresponding interviews in the lab (N=710). • Interviewers themselves filled in the questionnaire. • Application of 2 payment schemes: payment per hour and payment per interview
Procedure: Method • Analyzing differences between subgroups of interviewers and between real and falsified data • 3 subgroups: payment scheme (per hour vs per interview); interviewer’s gender (male vs female ); interviewer ’ optimism about future economic development (optimists vs pessimists) • 4 meta-indicators: acquiescent responding style (ARS); participation (past political activities); non-differentiation (ND); semi-open
Results: Interviewer effects Effects of payment scheme on meta-indicators payment per interview payment per hour t-value N mean sd N mean sd p (df=708) ARS false 361 0,47 0,12 349 0,45 0,12 1,728 0,084 real 377 0,52 0,13 333 0,51 0,13 0,505 0,614 participation false 361 0,36 0,16 349 0,38 0,17 1,340 0,181 real 377 0,42 0,16 333 0,45 0,16 2,835 0,005 non-differentiation false 361 0,905 0,018 349 0,910 0,017 3,724 0,000 real 377 0,900 0,017 333 0,899 0,016 0,501 0,617 semi-open false 361 0,021 0,074 349 0,022 0,072 0,304 0,761 real 377 0,048 0,109 333 0,047 0,103 0,178 0,859
Results: Interviewer effects Effects of interviewer's gender on meta-indicators male interviewer female interviewer t-value N mean sd N mean sd p (df=708) ARS false 202 0,48 0,12 508 0,46 0,12 1,757 0,079 real 204 0,54 0,12 506 0,51 0,13 2,596 0,010 participation false 202 0,38 0,18 508 0,37 0,16 1,086 0,278 real 204 0,44 0,15 506 0,43 0,16 1,040 0,299 non-differentiation false 202 0,906 0,017 508 0,908 0,018 1,480 0,139 real 204 0,902 0,017 506 0,899 0,016 2,433 0,015 semi-open false 202 0,031 0,086 508 0,018 0,067 2,156 0,031 real 204 0,051 0,110 506 0,046 0,104 0,666 0,506
Results: Interviewer effects Effects of interviewer's optimism on meta-indicators optimists pessimists t-value N mean sd N mean sd p (df=708) ARS false 389 0,45 0,12 321 0,48 0,12 2,963 0,003 real 387 0,52 0,13 323 0,52 0,13 0,065 0,949 participation false 389 0,39 0,17 321 0,35 0,16 2,666 0,008 real 387 0,44 0,16 323 0,42 0,16 1,468 0,143 non-differentiation false 389 0,9097 0,0173 321 0,9048 0,0180 3,676 0,000 real 387 0,8998 0,0169 323 0,8997 0,0159 0,099 0,921 semi-open false 389 0,024 0,076 321 0,019 0,069 0,963 0,336 real 387 0,039 0,093 323 0,057 0,119 2,179 0,030
Conclusion • Differences between real and falsified data depend on payment scheme and characteristics of the interviewer: Collecting as much information as possible about the interviewer. Considering the payment schemes. • Heterogeneity of meta-indicators: Considering cluster procedures for more than one cluster of falsifiers.
Thank you for your attention! Peter.Winker@wirtschaft.uni-giessen.de Natalja.Menold@gesis.org Uta.Landrock@gesis.org
Recommend
More recommend
