The Effects of Raking and Cell Phone Integration on BRFSS Outcomes - PDF document

4/24/2012 The Effects of Raking and Cell Phone Integration on BRFSS Outcomes Machell Town, M.S. Carol Pierannunzi, Ph.D. Division of Behavioral Surveillance Office of Surveillance, Epidemiology, and Laboratory Services Division of Behavioral Surveillance 1

4/24/2012 Brief Agenda  Weighting procedures  Design weights  Post stratification  Iterative proportional fitting  Why change weighting procedures now?  Cell phone  Computer capacity  Impact of changes on estimation  BRFSS  Examples of small and large impact  Changes when cell phones are incorporated  Conclusions  Brief look at state level phone use data (preliminary) 2

4/24/2012 WEIGHTING PROCEDURES 3

4/24/2012 Design and GeoStrata Weighting  Takes into account the geographic region/strata of the sample.  Design weight uses number of adults in household and number of phones in household for landline sample.  BRFSS landline sample is drawn using low/high density strata within each of the regions (usually smaller than states)  Stratum weight (_STRWT) = NRECSTR/ NRECSEL 4

4/24/2012 Calculating the Design Weight  Design Weight = _STRWT* (1/NUMPHON2) * NUMADULT  NUMPHON2= number of phones within the household  NUMADULT = number of adults eligible for the survey within the household  Questions for the design weights are asked in screening questions and in demographic sections of the survey 5

4/24/2012 Data Weighting  Data weights take the design weighting and incorporate characteristics of the population and the sample  Final Weights (_FINALWT) = Design Weight * some form of data weighting  In past BRFSS used post stratification  In 2008 Iterative Proportional Fitting was first used  In 2011 Iterative Proportional Fitting will be only method of data weighting for BRFSS 6

4/24/2012 Post -Stratification WEIGHTING 7

4/24/2012 Where We Have Been--- Post Stratification  Post Stratification is based on known demographics of the population.  For BRFSS Post stratification included: · Regions within states · Race/ Ethnicity (in detailed categories) · Gender · Age (in 7 categories)  Post-stratification forces the sum of the weighted frequencies to equal the population estimates for the region or state by race, age ,and gender.  Post stratification weights are applied to the responses, allowing for estimates of how groups of non- respondents would have answered survey questions.  8

4/24/2012 Post-stratification  Post-stratification Adjustment Factor is calculated for each race/ethnicity, gender, and age group combination.  Requires knowledge of each subset of each factor at the geographic level of interest – otherwise categories must be collapsed  Requires a minimum number of persons in each cell — otherwise categories must be collapsed  _POSTSTR = Population/Design weight within the weighting class cell. 9

4/24/2012 Weight Trimming  Sometimes post- stratification resulted in very small or disproportionately large weights within age/race/gender/region categories.  Weight trimming or category collapsing would be done if categories were disproportionately large or too small (< 50 responses). 10

4/24/2012 Iterative Proportional Fitting (Raking) WEIGHTING 11

4/24/2012 Iterative Proportional Fitting Rather than adjusting Region weights to Age by Race Age categories, IPF adjusts for Age by each Gender Race dimension separately in an iterative process. Gender Gender by Race The process will continue up to 75 times, or until data Marital Phone Status Type converges to Census Home Education Ownership estimates. 12

4/24/2012 New Variables Introduced as Controls With IPF  Education  Marital status  Home ownership/renter  Telephone source (cell phone or landline) 13

4/24/2012 Post Stratification vs. Iterative Proportional Fitting Iterative Post Proportional Stratification Fitting Operates with less Allows for computer time incorporation of new variables. Allows for incorporation of cell phone data. Seems to more accurately represent population data (reduces bias). 14

4/24/2012 Why Incorporate IPF Now?  Computer capacity has increased.  Cell phones are becoming larger percentage of the total number of calls.  Noncoverage with declining response rates makes weighting more important than ever. 15

4/24/2012 Examples of IPF From 2010 Data  Note that example may be slightly different from 2011 analyses because  We did not collect home ownership at that time  We still used phone interruption variable  Some of the iterations are different than will be conducted on 2011 dataset 16

4/24/2012 Raking – Iteration 1 Should be │ .025 │ Output or less Weight Sum of % of Sum of Target Weights Output Target % of Difference First Control Variable Weights Total Difference Weights Weights in % Age 18-24,Male 87122.60 95468 -8345.40 6.533 7.159 -0.626 Age 18-24,Female 77180.40 90249 -13068.60 5.788 6.768 -0.980 Age 25-34,Male 109419.36 118670 -9250.64 8.206 8.899 -0.694 Age 25-34,Female 114395.17 112007 2388.17 8.579 8.400 0.179 Age 35-44,Male 121328.71 117184 4144.71 9.099 8.788 0.311 Age 35-44,Female 115609.98 113779 1830.98 8.670 8.533 0.137 Age 45-54,Male 138658.26 127077 11581.26 10.398 9.530 0.869 Age 45-54,Female 136904.33 127439 9465.33 10.267 9.557 0.710 Age 55-64,Male 90338.77 95032 -4693.23 6.775 7.127 -0.352 Age 55-64,Female 91693.43 97422 -5728.57 6.876 7.306 -0.430 Age 65-74,Male 57475.54 54171 3304.54 4.310 4.062 0.248 Age 65-74,Female 62709.50 61828 881.50 4.703 4.637 0.066 Age 75+,Male 49772.58 46515 3257.58 3.733 3.488 0.244 Age 75+,Female 80867.37 76635 4232.37 6.064 5.747 0.317 17 17

4/24/2012 Raking – Iteration 1 Output Sum of % of Target % Weight Sum Target Weights Output of Difference Second Control Variable of Weights Total Difference Weights Weights in % WH NH 1151321.16 1156947 -5625.84 86.340 86.762 -0.422 OT NH 15305.42 12036 3269.42 1.148 0.903 0.245 HISP 85300.51 84230 1070.51 6.397 6.317 0.080 BL NH,AS NH,AI NH 81548.91 80263 1285.91 6.116 6.019 0.096 18 18

4/24/2012 Raking - Iteration 1 Input Sum of % of Weight Sum Target Weights Input Target % Difference Third Control Variable of Weights Total Difference Weights of Weights in % Less than HS 89962.05 143928 -53966.35 6.746 10.793 -4.047 HS Grad 412857.99 414505 -1646.81 30.961 31.085 -0.123 Some College 388163.96 448218 -60054.20 29.109 33.613 -4.504 College Grad 442492.00 326825 115667.37 33.183 24.509 8.674 19 19

4/24/2012 Raking – Iteration 1 Output Weight Sum of % of Target Target % of Difference Sum of Weights Output Fourth Control Variable Weights Total Difference Weights Weights in % Married 816399.38 792326 24073.29 61.223 59.418 1.805 Never married, member 277180.73 300111 -22930.01 20.786 22.506 -1.720 unmarried couple Divorced, Widowed, Separated 239895.88 241039 -1143.29 17.990 18.076 -0.086 Output Sum of % of Weight Sum Target Weights Output Target % of Difference Fifth Control Variable of Weights Total Difference Weights Weights in % Phone interruption 78558.62 82944 -4385.49 5.891 6.220 -0.329 No Phone Interruption 1254917.38 1250532 4385.49 94.109 93.780 0.329 20 20

4/24/2012 Raking – Iteration 1 Output Weight Sum of % of Sum of Target Weights Output Target % of Difference Sixth Control Variable Weights Total Difference Weights Weights in % Male, WH NH 553107.34 552171 936.34 41.479 41.408 0.070 Male, BL NH,AS NH,AI NH,OT 101008.49 101946 -937.51 7.575 7.645 -0.070 NH,HISP Female, WH NH 598213.82 604776 -6562.18 44.861 45.353 -0.492 Female, HISP 38304.69 32837 5467.69 2.873 2.463 0.410 Female, BL NH,AS NH,AI NH,OT 42841.66 41746 1095.66 3.213 3.131 0.082 NH 21 21

4/24/2012 Raking – Iteration 1 Output Weight Sum of % of Sum of Target Weights Output Target % of Difference Seventh Control Variable Weights Total Difference Weights Weights in % 18-34, WH NH 308020.95 332809 -24788.05 23.099 24.958 -1.859 18-34, BL NH,AS NH,AI NH,OT 80096.58 83585 -3488.42 6.007 6.268 -0.262 NH,HISP 35-54, WH NH 442299.71 421539 20760.71 33.169 31.612 1.557 35-54, BL NH,AS NH,AI NH,OT 70201.57 63940 6261.57 5.265 4.795 0.470 NH,HISP 55+, WH NH 401000.50 402599 -1598.50 30.072 30.192 -0.120 55+, BL NH,AS NH,AI NH,OT 31856.70 29004 2852.70 2.389 2.175 0.214 NH,HISP 22 22

4/24/2012 Raking – Iteration 1 Output Weight Sum of % of Sum of Target Weights Output Target % of Difference Eighth Control Variable Weights Total Difference Weights Weights in % Cell Phone Only 210390.11 197088 13302.35 15.778 14.780 0.998 Landline Only 270206.34 280297 -10090.31 20.263 21.020 -0.757 Landline and Cell Phone 852879.55 856092 -3212.04 63.959 64.200 -0.241 23 23