Study of Error in Survey Reports of Move Month Using USPS Change of - PowerPoint PPT Presentation

Study of Error in Survey Reports of Move Month Using USPS Change of Address Records Mary H. Mulry, Elizabeth M. Nichols, Jennifer Hunter Childs U.S. Census Bureau 2014 Joint Statistical Meetings August 4, 2014

Motivation for study • Surveys ask respondents to recall events • U.S. censuses ask respondents to recall where they lived on Census Day, April 1 • Some census evaluation interviews may occur 11 months after April 1 • Assumption: respondents who move around April 1 are able to give correct answers • Recall errors may cause respondents to be enumerated at the wrong location 2

Literature on Event Recall • Much research on respondent recall and several models of survey response process Cannell et al. (1981), Sudman et al. (1996) • Telescoping may be backward or forward Rubin & Baddeley (1989), Huttenlocher et al. (1990); Janssen et al. (2006) • Wording of questions may aid recall Prohaska et al. (1998), Tourangeau et al. (2000) • Use of move date as anchor may aid recall Wagenaar (1986), Belli et al. (2008) • Could not find research on error in recalling move dates 3

Source of sample of movers • U.S. Postal Service National Change of Address database – Name of person making request – Current address where mail delivered – Destination address where mail will be forwarded – Date to start forwarding mail – Indicator of whether a Family move or an Individual move 4

Recall Bias Survey (RBS) asked move months • US Census Bureau conducted RBS as research project during 2010 Census (Census Day: April 1) – Questionnaire very similar to one used in evaluations Linse, Pape, Rosenberger, Contreras (2012) • Sample of addresses selected from NCOA file – Requests submitted in March & April 2010 – Linked to commercial database to get telephone number 5

Recall Bias Survey (RBS) implementation • Questions about moves: – Did HH member move in 2010? – If yes, asked for addresses and dates of all moves • 3 waves of telephone interviews: 4,500 each – June: 2 to 3 months after Census Day, RR 69% – Sept: 5 to 6 months after Census Day, RR 66% – Feb: 10 to 11 months after Census Day, RR 63% 6

Research strategy • Assume NCOA record is “truth” • Compare each RBS response to NCOA record to be sure household has same person and mentions same address • Investigate error in reports of moves and move month – Logistic regression models – Multinomial analysis 7

Limitations • NCOA undercoverage: Many movers do not file a request • NCOA overcoverage: Some requests do not represent a move • Movers may forward mail to one address but move to another • Only 20% of addresses link to a telephone number 8

Data for analysis • 8,937 respondents combining 3 waves • 3,424 were same person as NCOA record – Address matched NCOA forwarding address – HH member matched NCOA name • Of those, 1,740 were same event as NCOA – Reported move to NCOA forwarding address – Reported date of that move 9

Logistic regression models • Variables studied – RBS reported a move – RBS reported move month agreed with NCOA – RBS reported move month indicated a Census Day address that agreed with NCOA • Models found association with – Length of time since move – Move-HH type • 1-person HH, Family move, Individual move 10

Error rate in reports of a move (% of NCOA moves not reported) by HH-move type by time since move 60% 53% 52% 50% 40% 40% 38% 40% 33% 30% 20% 10% 0% 2 - 3 5 - 6 10 - 11 1 Person HH Family Individual months months months move move move 11

Error rate in reports of move month (% with reported move month disagreeing with NCOA) by time since move by HH-move type 60% 50% 40% 35% 33% 30% 27% 24% 23% 17% 20% 10% 0% 2 - 3 5 - 6 10 - 11 1-Person HH Family Individual months months months move move move 12

Error rate in reported Census Day residence ( % with reported CD residence disagreeing with NCOA) by time since move by HH-move type 60% 50% 40% 30% 22% 21% 20% 15% 15% 14% 11% 10% 0% 2 - 3 5 - 6 10 - 11 1-person HH Family Individual months months months move move move 13

Effect of variables • Reports given 10 to 11 months after move had highest error rates for all variables – No difference for reports between 2 to 3 months and 5 to 6 months after move • Reports about moves by an Individual in HH had highest error rates for all variables – Family moves more likely to be reported than 1-person HH moves – No difference in error rates for reported move month for Family and 1-person HH moves 14

Multinomial analysis • Direction of net effect of error in RBS reports move month – Zero • Errors offset, equally earlier and later than NCOA – Backward telescoping • More errors tend to be earlier than NCOA – Forward telescoping • More errors to be later than NCOA • Data: self-responses 15

Direction of error in self-reports of move month 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 2 to 3 months 5 to 6 months 10 to 11 months 16

Direction of error • Self-responses – Backward telescoping occurred 10 to 11 months after a move, but not at 2 to 3 months or 5 to 6 months afterwards – Differ from Janssen et al. (2006) about when backward telescoping starts 17

Future research • More research needed to identify point when error in recall of move month becomes problematic • Complete address and strong matching criteria necessary for definitive results 18

Contact mary.h.mulry@census.gov elizabeth.may.nichols@census.gov jennifer.hunter.childs@census.gov 19