CHIS 2013 Sample Design and Survey Methodology TAC August 30, 2012 1 - PowerPoint PPT Presentation

CHIS 2013 Sample Design and Survey Methodology TAC August 30, 2012 1

Welcome  Ninez Ponce, CHIS PI  David Grant, CHIS Director  A. A. Afifi, TAC Chair 2

Introductions (In order of Adobe Connect roster) 1) Name 2) Where you work 3) Expertise 3

Meeting Goals  Overview of CHIS methodology past and present  Discuss methodological topics most relevant to CHIS  Sketch out design and experiment ideas as they arise  No burden to leave today with clear solutions to specific challenges…work groups may follow 4

Meeting Themes  CHIS Field Schedule  Methodological experiments and implementing methodological changes?  Interview Length  Shortening interview length; What role does questionnaire length play in overall survey cost and data quality?  Measurement  Improve measurement maintaining data series? 5

Costs and Cost/Error Tradeoffs  Adapt CHIS methodology to reduce overall survey costs  Proactive consideration (e.g., experiments and tests that will reduce costs long ‐ term)  Some cost ‐ savings incur up ‐ front costs  Most cost decisions are cost/quality trade ‐ offs  Further adaptive/responsive design approach to data collection 6

The Interview  Telephone survey on wide range of health issues (see questionnaire content handouts)  Only mode for all data collection  Landline ‐ RDD, Cell ‐ RDD, Surname List  Pre ‐ notification letters with $2 incentive when address matched from records  Westat interviewers conduct interviews from call centers and home 7

Sample designed to represent…  Residents of CA  Stratification by CA counties and county groups  In 2011, 44 strata representing 58 counties  Most with targets of n=600 ea.  Smallest counties (17) in 3 grouped strata  Key ethnic and racial subgroups  Initial proportionate allocation accounts for this  Vietnamese and Korean oversample since 2001  Am Indian/Alaskan Native in 2001 and “current” 8

Field Schedule  Every two years since 2001  In past, data collected w/in approx. 9 ‐ month window  2011 ‐ 12 forward in quarterly “rolling” sample starting in January of 2 ‐ year cycle  T4 currently in field, finishing end of Dec. 9

TCE Building Health Communities  14 geographically defined communities in CA  Chosen by TCE as part of 10 ‐ year health improvement effort  CHIS data collection is baseline and follow ‐ up measurement (i.e., “panel in development”)  CHIS 2009 oversampled numbers in these areas  Expansion of sample to insure 400 completes per site  Participants re ‐ contacted in 2012 and asked about future participation (n  100 per site)  http://www.calendow.org/healthycommunities/  http://www.healthpolicy.ucla.edu/healthprofiles09.aspx 10

Status of 2013 Planning  SDSM TAC is one of 5 TACs  Questionnaire being revised based on other TACs  Room for experiments resulting from today  Sample design  Room for adjustments based on this meeting  Increase/decrease in cell sample; alternative sampling methods for hard ‐ to ‐ reach populations (e.g., RDS) 11

CHIS Performance Over Time Royce Park 12

CHIS Response Rates 2001 2003 2005 2007 2009 Screener Interview 59.2 55.9 49.8 35.6 36.1 Extended Interview Household ‐ ‐ 59.3 59.4 54.7 Adult 63.7 59.9 54.0 52.8 49.0 Child 87.6 81.4 75.2 73.7 72.9 Adolescent 63.5 57.3 48.5 44.1 42.8 Overall Household ‐ ‐ 29.6 21.1 19.7 Adult 37.7 33.5 26.9 18.7 17.7 Child 33.0 27.3 25.2 16.8 15.7 Adolescent 23.9 19.2 14.2 10.2 7.9 * Landline/List Sample (excludes cell phone sample), Weighted, AAPOR RR4 13

CHIS Response Rates 90 80 Extended Child 70 Response Rate (%) 60 Extended Household 50 Extended Adult Extended Teen 40 Screener Interview 30 Overal Household 20 Overall Adult Overall Child 10 Overall Teen 0 2001 2003 2005 2007 2009 14 * Landline/List Sample (excludes cell phone sample), Weighted, AAPOR RR4

Screener & Extended Response Rates 90 80 Extended Child 70 Response Rate (%) 60 Extended Household 50 Extended Adult 49.8 Extended Teen 40 35.6 Screener Interview 30 20 10 0 2001 2003 2005 2007 2009 15 * Landline/List Sample (excludes cell phone sample), Weighted, AAPOR RR4

Overall Response Rates 90 80 70 Response Rate (%) 60 50 40 Screener Interview 30 Overal Household 20 Overall Adult Overall Child 10 Overall Teen 0 2001 2003 2005 2007 2009 16 * Landline/List Sample (excludes cell phone sample), Weighted, AAPOR RR4

17 CHIS 2009 Screener Response Rate by Strata Response Rate (%) * Landline/List Sample (excludes cell phone sample), Weight, AAPOR RR4 25 30 35 40 45 50 San Francisco Santa Clara Solano San Mateo Los Angeles Ventura San Diego Riverside Contra Costa Lake El Dorado Fresno San Joaquin Sacramento Orange Nevada CA County Alameda Imperial Santa Barbara Yolo San Bernardino Placer Kings Marin Sonoma Stanislaus Napa Mendocino Merced Madera Monterey Overall State RR = 36.1 Yuba Kern Santa Cruz Sutter San Benito Tulare Sierra Balance San Luis Obispo Shasta Butte Colusa, Glenn, Tehama Humboldt North Balance

18 CHIS 2009 Screener Response Rate by Strata Response Rate (%) * Landline/List Sample (excludes cell phone sample), Weight, AAPOR RR4 25 30 35 40 45 50 San Francisco Santa Clara Solano San Mateo Los Angeles Ventura San Diego Smallest Counties Riverside Contra Costa Lake El Dorado Fresno San Joaquin Sacramento Orange Nevada CA County Alameda Imperial Santa Barbara Yolo San Bernardino Placer Kings Marin Sonoma Stanislaus Napa Mendocino Merced Madera Monterey Overall State RR = 36.1 Yuba Kern Santa Cruz Sutter San Benito Tulare Sierra Balance San Luis Obispo Shasta Butte Colusa, Glenn, Tehama Humboldt North Balance

19 CHIS 2009 Screener Response Rate by Strata Response Rate (%) * Landline/List Sample (excludes cell phone sample), Weight, AAPOR RR4 25 30 35 40 45 50 San Francisco Santa Clara Solano San Mateo Los Angeles Ventura San Diego Medium Counties Riverside Contra Costa Lake El Dorado Fresno San Joaquin Sacramento Orange Nevada CA County Alameda Imperial Santa Barbara Yolo San Bernardino Placer Kings Marin Sonoma Stanislaus Napa Mendocino Merced Madera Monterey Overall State RR = 36.1 Yuba Kern Santa Cruz Sutter San Benito Tulare Sierra Balance San Luis Obispo Shasta Butte Colusa, Glenn, Tehama Humboldt North Balance

20 CHIS 2009 Screener Response Rate by Strata Response Rate (%) * Landline/List Sample (excludes cell phone sample), Weight, AAPOR RR4 25 30 35 40 45 50 San Francisco Santa Clara Solano San Mateo Los Angeles Ventura San Diego Largest Counties Riverside Contra Costa Lake El Dorado Fresno San Joaquin Sacramento Orange Nevada CA County Alameda Imperial Santa Barbara Yolo San Bernardino Placer Kings Marin Sonoma Stanislaus Napa Mendocino Merced Madera Monterey Overall State RR = 36.1 Yuba Kern Santa Cruz Sutter San Benito Tulare Sierra Balance San Luis Obispo Shasta Butte Colusa, Glenn, Tehama Humboldt North Balance

Adult Response Rate by Sample Adult Characteristics Landline/List Cell phone Total 49.0 56.2 Sex Male 43.7 56.3 Female 53.8 56.1 Age 18 to 30 years 36.0 52.3 31 to 45 years 42.6 57.6 46 to 65 years 52.1 57.0 Over 65 years 60.4 64.3 Type of household With child and/or Adolescent 43.3 52.1 Without child or Adolescent 53.1 59.5 Number of adults in household 1 65.2 56.3 2 52.6 59.7 3 or more 39.5 51.3 Adult was screener respondent Yes 64.2 58.9 No 34.5 20.9

How do we compare to other surveys  RR’s difficult to compare due to disposition definitions, RR calculations, and undocumented differences between surveys  Comparable to CA BRFSS over the years  27.1% CA BRFSS RR v. 19.7  BRFSS has single adult interview, CHIS considers any one interview (Adult, Teen, Child) a “complete” 22

Address ‐ based Sampling Pilot Matt Jans 23

ABS – Address ‐ based Sampling  Pro: More complete coverage of HH population than telephone frames  Con: Unless survey questionnaire is mail, requires switch to another mode for completion  Challenge: How to obtain phone number, and eventually completed phone surveys, from sampled addresses 24

ABS Pilot Design  Implemented in 2 BHC sites  Selected for lower survey saturation  Spanish language prevalence  “Short Questionnaire” (i.e., screener form) mailed to all addresses  Requests adult response  A few basic health and demography questions and  Data expected end ‐ of ‐ year 25

Test of Messaging and Outreach  Random half in each site receive at 2 nd and 3 rd mailing…  TCE logo on the out ‐ going envelope  Insert showing TCE/BHC support (1/3 page slip in bright color or with color logo)  Seeking support of Program Managers (PMs) in the two communities  Vouch for our survey with their members  Distribute info via email list, etc. 26