Designing a Multipurpose Designing a Multipurpose Longitudinal - PowerPoint PPT Presentation

Designing a Multipurpose Designing a Multipurpose Longitudinal Incentives Experiment for the SIPP Ashley Westra U.S. Census Bureau* U.S. Census Bureau DC-AAPOR & WSS: Summer Conference Preview/Review 2015 August 3 2015 August 3, 2015 *Any views expressed are those of the author and not necessarily those of the U.S. Census Bureau.

Overview Overview  Background/Previous SIPP Incentive Experiments p  SIPP 2014 Experiment  Design D i  Wave 1 Results  Wave 2 Goals  Future Plans Future Plans

The SIPP Survey The SIPP Survey  The Survey of Income and Program Th S f I d P Participation (SIPP) is a demographic longitudinal survey that collects data and longitudinal survey that collects data and measures change for many topics, including:  Economic Well-being Eco o c We be g  Family Dynamics  Education  Assets  Health Insurance  Childcare  Food Security

The SIPP Survey Design The SIPP Survey Design  Previous Panels (1996, 2001, 2004, 2008)  3-5 year panels y p  Conducted in waves, each 4 months long  4 equally sized rotation groups  4 equally sized rotation groups  2014 Panel  4 year panel  Conducted in waves, each 1 year long Conducted in waves, each 1 year long  No rotation groups

Previous SIPP Incentive Experiments Previous SIPP Incentive Experiments  Since the 1996 Panel, SIPP has conducted several incentive tests of different types. yp  Designed to test the effect of monetary incentives on overall response rates incentives on overall response rates.

Previous SIPP Incentive Experiments Previous SIPP Incentive Experiments  Tested both conditional and unconditional  Tested both conditional and unconditional incentives.  Tested both random assignment as well as discretionary incentives discretionary incentives  Experimented with the monetary amount of the incentive, with $10, $20, and $40 f th i ti ith $10 $20 d $40 being the typical choices.

Results of Previous Experiments Results of Previous Experiments 1996 P 1996 Panel l  $20 (but not $10) unconditional incentives $20 (but not $10) unconditional incentives were effective in reducing household nonresponse in Wave 1 and this effect nonresponse in Wave 1, and this effect remained in later waves.

Results of Previous Experiments Results of Previous Experiments 2001 Panel  For 7 out of 9 waves $40 conditional  For 7 out of 9 waves, $40 conditional discretionary incentives increased response rates.

Results of Previous Experiments Results of Previous Experiments 2004 Panel  Households that receive $40 discretionary H h ld th t i $40 di ti incentives are more likely to receive them in later waves.

Results of Previous Experiments Results of Previous Experiments 2008 Panel  The Wave 1 $20 unconditional incentive $ effectively improved response rates in Waves 1-3 by 1.1-1.4% compared to the control.  The discretionary $40 conditional incentive (in any wave) had an effect in Waves 7-9, i improving response rates by 1.6-3.1% i t b 1 6 3 1% compared to the control.

2014 Panel 2014 Panel – Experimental Design Experimental Design  Households randomly Households randomly put into 1 of 4 equally Group Wave 1 Wave 2 sized groups ( ≈ 13,000 1 1 $0 $0 $0 $0 households).  Conditional incentives 2 $0 $40 are distributed as debit di t ib t d d bit 3 $20 $0 cards by NPC.  Testing the use of a  Testing the use of a 4 4 $40 $40 (a) $40 (a) $40 propensity model to (b) $0 assign incentives in later g waves.

2014 Panel Wave 1 2014 Panel Wave 1 – Results Results  Households were Group Wave 1 randomly assigned to $0, y g 1 1 $0 $0 $20, or $40 conditional 2 $0 incentives incentives. 3 $20  $20 increased the response rate by 1 2% rate by 1.2% 4 4 $40 $40  $40 increased the response rate by 3.5% b 3 5%

2014 Panel Wave 1 2014 Panel Wave 1 – Results Results Poverty Stratum While While Response Rates Distribution of Interviewed Incentive Low Non-Low Incentive Low Non-Low incentives Group Income Income Group Income Income 61% affected affected $0 71% 66% $0 38% 62% $20 $20 73% 73% 67% 67% $20 $20 39% 39% 61% response rates, $40 76% 68% $40 39% 61% 61% they did not ALL 72% 67% ALL 39% they did not affect the Urban/Rural Response Rates Distribution of Interviewed distribution of distribution of Incentive Incentive Incentive Incentive the Group Urban Rural Group Urban Rural $0 67% 71% $0 80% 20% 19% interviewed. interviewed. $20 68% 72% $20 81% $40 70% 74% $40 80% 20% ALL 68% 72% ALL 80% 20%

2014 Panel Wave 2 2014 Panel Wave 2 – Model Model  Create a logistic regression model predicting the probability of response given certain household characteristics f i i h h ld h i i  Census Region Group Wave 1 Wave 2  Age of Householder g  Gender 1 $0 $0  Race 2 $0 $40  Hispanic Origin p g 3 $20 $0  Education  Marital Status 4 $40 (a) $40  Income Income (b) $0 $  Work Status  Assign incentives based on the predicted probabilities to improve coverage improve coverage.

2014 Panel Wave 2 2014 Panel Wave 2 – Tests Tests  Effect of randomly Effect of randomly Group Wave 1 Wave 2 assigned incentives on 1 $0 $0 response rates response rates 2 $0 $40  Does the Wave 1 incentive effect carry-over to Wave ff t t W 3 3 $20 $20 $0 $0 2? 4 $40 (a) $40  4(b) vs. 1 (b) $0  3 vs. 1

2014 Panel Wave 2 2014 Panel Wave 2 – Tests Tests  Effect of randomly Effect of randomly Group Wave 1 Wave 2 assigned incentives on 1 $0 $0 response rates response rates 2 $0 $40  What is the effect of duplicate incentives? d li t i ti ? 3 3 $20 $20 $0 $0  4(a) vs. 1 4 $40 (a) $40  4(a) vs. 4(b) (b) $0

2014 Panel Wave 2 2014 Panel Wave 2 – Tests Tests  Effect of randomly Effect of randomly Group Wave 1 Wave 2 assigned incentives on 1 $0 $0 response rates response rates 2 $0 $40  What is the effect of a later incentive? i ti ? 3 3 $20 $20 $0 $0  2 vs. 1 4 $40 (a) $40 (b) $0

2014 Panel Wave 2 2014 Panel Wave 2 – Tests Tests  Effectiveness of the propensity model in assigning incentives model in assigning incentives, Gro p Wa e 1 Wa e 2 Group Wave 1 Wave 2 conditional on Wave 1 1 $0 $0 incentives.  For a given percentage of 2 $0 $40 households assigned a model- based incentive compare the based incentive, compare the 3 3 $20 $20 $0 $0 distributions and response 4 $40 (a) $40 rates of:  2 vs. 1 (b) $0  4(a) vs. 4(b)

2014 Panel Wave 3 Plans 2014 Panel Wave 3 Plans  Depending on the results of Wave 2, we may decide to implement the propensity y p p p y model.  We are concerned that the group of  We are concerned that the group of households that received $40 incentives f for two consecutive waves will expect i ill them again. g

Thank you! Contacts: Ashley.M.Westra@census.gov Mahdi S Sundukchi@census gov Mahdi.S.Sundukchi@census.gov