DC-AAPOR/WSS SUMMER CONFERENCE 2019 INCREASING REPRESENTATIVENESS - PowerPoint PPT Presentation

DC-AAPOR/WSS SUMMER CONFERENCE 2019 INCREASING REPRESENTATIVENESS THROUGH THE USE OF PREDICTIVE MODELING AND TARGETED OUTREACH A.L. DJANGALI, J. JOSEPH-DAVID, & L. TROFIMOVICH

DC-AAPOR/WSS SUMMER CONFERENCE 2019 STUDY BACKGROUND ▸ Kidney Disease Quality of Life Survey (KDQOL) ▸ Patient-reported outcomes on quality of life measures for Medicare beneficiaries with end-stage renal disease (ESRD) ▸ Survey conducted yearly as part of a larger evaluation of an alternative payment model

DC-AAPOR/WSS SUMMER CONFERENCE 2019 SURVEY METHOD ▸ Conducted from 2016 to 2018 ▸ Two overlapping censuses in 2016 (≈ 13,511 each) ▸ Sample in 2017 (n=17,198) ▸ Sample in 2018 (n=18,122) ▸ Mail-CATI sequential mode design 1 st Survey 2 nd Survey Pre-note letter Outbound CATI • Toll-free RHL • Separate • Outbound • Transfer to English & CATI with bilingual staff Spanish undeliverable as needed materials mail cases ▸ Conducted in English and Spanish

DC-AAPOR/WSS SUMMER CONFERENCE 2019 2016 RESPONSE ANALYSIS After 2016 Census: ▸ Logistical regression model to identify which socio-demographic characteristics would predict response ▸ African-Americans, Hispanics, 18-64 year olds: ▸ Less likely to respond overall ▸ More likely to complete by phone than mail ▸ More likely to complete over phone after making contact ▸ African-Americans and Hispanics disproportionally impacted by ESRD Hubbard, F.A., Weiss, C., & Djangali, A.L. (2017). Modeling Contact and Cooperation in an Administrative Record Based Mail/CATI Survey. Paper presented at the 2017 American Association for Public Opinion Research (AAPOR) conference, New Orleans, Louisiana.

DC-AAPOR/WSS SUMMER CONFERENCE 2019 CHANGES IMPLEMENTED FOR 2017 & 2018 ▸ Started CATI follow-up 2 weeks earlier for hard-to-reach (HTR) groups ▸ Used Census tract data to identify sample members more likely to be Hispanic ▸ Identified previous CATI completers Djangali, A.L., Joseph-David, J., & Hubbard, F.A. (2018). Methods for increasing response rates for Medicare beneficiaries. Poster presented at the 2018 American Association for Public Opinion Research (AAPOR) conference, Denver, Colorado. DOI: 10.13140/RG.2.2.31912.67845

DC-AAPOR/WSS SUMMER CONFERENCE 2019 CHANGES IMPLEMENTED FOR 2017 & 2018 ▸ 2016 1 st Survey 2 nd Survey Pre-note letter Outbound CATI • Toll-free RHL • Separate • Outbound • Transfer to English & CATI with bilingual staff Spanish undeliverable as needed materials mail cases ▸ 2017 1 st Survey 2 nd Survey Pre-note letter Outbound CATI & 2018 • Toll-free RHL • Bilingual tickler • Outbound • Transfer to • Bilingual tickler message CATI with bilingual staff • Double-stuffed undeliverable as needed message mail cases • Proactively • Outbound calls English & • Bilingual tickler Spanish assign cases to previous materials message to bilingual CATI • Double-stuffed staff completers materials • Early outbound calling for HTR

DC-AAPOR/WSS SUMMER CONFERENCE 2019 RESPONSE RATE COMPARISONS † 2016 2017 2018 0.5 43% 0.45 42% 41% 40% 38% 38% 0.4 36% 34% 34% 0.35 31% 28% 28% 0.3 0.25 0.2 0.15 0.1 0.05 0 Overall 18-54 Black/Afr. Amer. Hispanic † AAPOR RR3

DC-AAPOR/WSS SUMMER CONFERENCE 2019 SIGNIFICANCE TESTING ▸ 2-sample t-tests of mean scores to control for sample size differences ▸ Significance (p<.05) across all subgroups 2016 2017 2018 Demographics Mean SE Mean SE Mean SE n n n 18-54 3,588 28.3% 0.75% 4,627 33.9% 0.70% 4,601 30.9% 0.68% Black/Afr. Amer. 6,042 34.3% 0.61% 7,487 39.5% 0.57% 7,380 37.6% 0.56% Hispanic 909 27.9% 1.49% 1,019 35.6% 1.50% 1,079 40.6% 1.49% 18-54 Black/Afr. Amer. Hispanic * *** *** 2016 vs 2017 *** * *** 2017 vs 2018 * ** *** 2016 vs 2018 * p<.05 ** p<.01 *** p<.001

DC-AAPOR/WSS SUMMER CONFERENCE 2019 FINDINGS & RECOMMENDATIONS ▸ Providing preferred language can help to increase representation among Hispanic respondents ▸ Increasing outreach attempts to traditional HTR respondents helps to increase response rates for those groups ▸ Small, data-driven changes can make a big impact

DC-AAPOR/WSS SUMMER CONFERENCE 2019 QUESTIONS? Please feel free to contact me with questions or to receive a copy of this presentation: Amy L. Djangali Senior Research Director, IMPAQ adjangali@impaqint.com