Exploring the Effect of an Advance Letter on Response and Eligibility Rates: A M Met eta-Ana nalysis S Stud udy for the he Nationa nal I Immuni unization S n Sur urvey (NIS) Abera Wouhib 1 , Jie Zhao 2 , Meena Khare 1 and Vicki Pineau 2 AAPOR , Orlando, FL May 18, 2012 1 National Center for Health S tatistics, Centers for Disease Control and Prevention 2 NOR C at the University of Chicago “The findings and conclusions in this paper are those of the authors and do not necessarily represent the views of the National Center for Health S tatistics, Centers for Disease Control and Prevention.”
Outline Introduction Methods Results Discussion
Introduction The National Immunization Survey (NIS) sponsored by the National Center for Immunization and Respiratory Diseases (NCIRD) conducted jointly by NCIRD and the National Center for Health Statistics (NCHS), Centers for Disease Control and Prevention(CDC) one of the largest national telephone surveys began data collection since 1994 monitor childhood vaccination for children 19 and 35 months living in the United States at the time of the interview The NIS is conducted as stratified, two phase surveys The first phase a list-assisted, random-digit-dialing survey to identify households (HH) with age-eligible children with stratification at state and local areas to ensure comparable precision among the state and local areas The second phase is a mailed survey to providers identified during the telephone interview to collect provider-reported vaccination histories
Introduction….cont’d Response rates in telephone surveys, including the NIS, continue to decline over time The NIS uses the Council of American Survey Research Organizations(CASRO ) response rate defined as a product of the resolution rate, screener rate and interview completion rate The NIS eligibility rate, which is defined as percentile of HHs with age eligible children among successfully screened-in HHs, has been also declining recently The differential in decline of CASRO response rates and eligibility rates has been wider among households(HHs) by advance letter mailing status in the past few years
Key Monitoring Statistics and Response Rates by Advance Letter Status, NIS 2005 -2010 Row Key Indicator 2005 2006 2007 2008 2009 2010 Random-digit-dialed phase AL WAL AL WAL AL WAL AL WAL AL WAL AL WAL 1 Total selected telephone # released 1460066 3005195 1645109 3392721 1469436 3069931 1760771 3950032 1579190 4731439 1711459 5366202 2 Total phone numbers released to CATI 1460066 1134024 1645109 1205815 1469436 1082296 1760771 1456188 1579190 1796841 1711459 2106383 3 Advance letters mailed 1460066 1645109 1469436 1760771 1579190 1711459 (row 3 AL /(row 2 AL + row 2 WAL )) 56.28% 57.70% 57.59% 54.73% 46.48% 44.83% 4 Resolved phone numbers 1095966 2625258 1215413 2981829 1063607 2699406 1238565 3459522 1080840 4147360 1164813 4732912 Resolution rate 75.06% 87.36% 73.88% 87.89% 72.38% 87.93% 70.34% 87.58% 68.44% 87.66% 68.06% 88.20% 5 HH identified 821225 263815 889909 247797 766851 207735 846609 261882 798106 316564 856624 328595 (row 5/row 4) 74.93% 10.05% 73.22% 8.31% 72.10% 7.70% 68.35% 7.57% 73.84% 7.63% 73.54% 6.94% 6 HHs screened for age eligible children 766354 240081 812099 216974 697688 181519 771047 229793 743833 286543 790893 293528 Screening completion rate (row 6/row 5) 93.32% 91.00% 91.26% 87.56% 90.98% 87.38% 91.07% 87.75% 93.20% 90.52% 92.33% 89.33% 7 HHs with no age-eligible children 743098 231412 786730 208383 676987 174413 750074 221088 724726 276737 773168 283848 (row 7/row 6) 96.97% 96.39% 96.88% 96.04% 97.03% 96.09% 97.28% 96.21% 97.43% 96.58% 97.76% 96.70% 8 HHs with age-eligible children 23256 8669 25369 8591 20701 7106 20973 8705 19107 9806 17725 9680 Eligibility rate (row 8/row 6) 3.03% 3.61% 3.12% 3.96% 2.97% 3.91% 2.72% 3.79% 2.57% 3.42% 2.24% 3.30% 9 HHs with with completed RDD interviews 19817 7050 22022 7043 18250 5883 18160 7097 16287 7781 15132 7783 interview completion rate (row 9/row 8) 85.21% 81.32% 86.81% 81.98% 88.16% 82.79% 86.59% 81.53% 85.24% 79.35% 85.37% 80.40% 10 CASRO RR (row 4 * row 6 * row 9) 59.69% 64.65% 58.53% 63.09% 58.06% 63.61% 55.47% 62.65% 54.37% 62.96% 53.64% 63.35% AL L – househ ehold mai ailed ed an an ad advan ance e let etter er WAL– ho hous useho hold witho hout ut a an a n advance let etter er
Trends in Response Rates by Advance Letter Status, NIS 2005 -2010
Trends in Eligibility Rates, NIS 2005 - 2010
Methods Our objective is to explore the association of an advance letter status and reporting no age eligible children in HHs because of its effect on both CASRO response and eligibility rates CASRO RR= (resolution rate) x (screening rate) x (completion rate) Screening Rate = HHs Screened−in HHs Identified x100 Interview Completion Rate = HH with completed RDD interviews x100 HHs with age eligible children Eligibility Rate = HHs with age eligible children x100 HHs Screened−in HHs identified (n) = HHs screened out + HHs with no age eligible + HHs with age eligible X Y Z 1 = 𝑌 𝑜 + 𝑍 𝑜 + 𝑎 𝑜 = 𝑄 1 + 𝑄 2 + 𝑄 3 , where 𝑄 1 , 𝑄 2 and 𝑄 3 are proportions
Methods ….cont’d By splitting Y as 𝑍 𝐵𝐵 for HHs with advance letter and 𝑍 𝑋𝐵𝐵 for HHs without advance letter, we computed the respective proportions 𝑄 𝐵𝐵 and 𝑄 𝑋𝐵𝐵 among identified HHs with and without advance letter To measure the association of an advance letter status and the reporting of no age eligible children, we use a relative risk, say RR = 𝑄 𝐵𝐵 𝑋𝐵𝐵 , 𝑄 RR > 1 implies direct association RR < 1 implies inverse association RR ≈ 1 indicates no association Clearly, RR is a random Variable and for convenience can be transformed in logarithmic form as: 𝜈 � 𝑗 = log RR = log 𝑄 𝐵𝐵 – log 𝑄 𝑋𝐵𝐵 for computational simplicity � and its variance can The log RR may be estimated from the data as 𝜄 be estimated using method of large sample approximation
Methods ….cont’d Let 𝜈 � 𝑗 be the log RR for each subgroup of interest by applying a methods of meta-analysis, where i=1,2,….k are the subgroups of a study group 𝜈 � 𝑗 is an effect size assumed to be independently distributed for 2 estimated by a method of i=1,2,….k, with an associated variance, 𝜏 𝑗 large sample approximation In meta-analysis, a one-way random-effects model can be stated as: 𝜈 � 𝑗 = 𝜄 + 𝜁 𝑗 + 𝜉 𝑗 , 𝜁 𝑗 − within study error and 𝜉 𝑗 − between study error 2 ), 𝜉 𝑗 ~ 𝑂 (0, 𝜐 2 ) and 𝜁 𝑗 ⟘ 𝜉 𝑗 for all i assuming that 𝜁 𝑗 ~ 𝑂 (0, 𝜏 𝑗 The assumption of random-effects modeling implies the existence of the between study errors and the parameter 𝜐 2
Methods ….cont’d We used the recent three years 2008 - 2010 NIS and RRD sample frame (exchange level) variables received from the Marketing Systems Group (MSG) to create two study groups 4 Regions by year 5 HH income subgroups by year The two groups each made up of independent subgroups within study group are: 12 subgroups of region by year , k = 12 15 subgroups of income by year , k = 15 In the region by year study group, the proportions for subgroup region south and 2008 are estimated as 𝐵𝐵 = HHs reported no age eligible children and with AL in South − 2008 𝑄 HHs identified with AL in South − 2008 𝑄 𝑋𝐵𝐵 = HHs reported no age eligible children and without AL in South − 2008 HHs identified without AL in South − 2008
Results Region-Y ear Subgroup µ � 𝑗 𝑇𝑇 ( µ � 𝑗 ) µ � 𝑗 [95 % CI of µ � i ] Wt. in % Region Y ear
Results Income by Y ear Subgroup µ � 𝑗 𝑇𝑇 ( µ � 𝑗 ) µ � 𝑗 [95 % CI of µ � i ] Wt. in % Income Y ear
Results…..cont’d � 𝑗 is positive and the 95% CI is also positive. This In all subgroups, 𝜄 implies a direct association among the two groups, the HHs with mailed advance letter and reporting no age eligible children � = 0.04 (RR=1.041) and the 95% CI In all cases the overall estimate, 𝜄 indicates that there is direct (positive) relationship among HHs with mailed advance letter and reporting no age eligible children Among region-year study group, South and West are more likely to report no eligible children higher than 4.1% (the overall mean) and Northeast is more likely to report less than the overall mean Among income-year group HHs with less than $25K annual income are less likely to report no age eligible children than HHs in other income groups
Recommend
More recommend
