Understanding Variance Estimator Bia ias in in Stratified - PowerPoint PPT Presentation

Understanding Variance Estimator Bia ias in in Stratified Two-Stage Sampling Khoa Dong 1 , Tim Trudell 1 , Yang Cheng 1 , Eric Slud 1,2 1 U.S. Census Bureau, 2 University of Maryland Joint Statistical Meetings Vancouver, CA July 29, 2018 1

Disclaimer This presentation is intended to inform interested parties of ongoing research and to encourage discussion of work in progress. Any views expressed on statistical, methodological, technical, or operational issues are those of the authors and not necessarily those of the U.S. Census Bureau. 2

Outline 1. Motivation 2. Overview of Current Population Survey (CPS) 3. Problem description 4. CPS variance estimation 5. Simulation results 3

Motivation • When estimating response rate 𝑞 and 𝑤𝑏𝑠(𝑞) for households in CPS non-self-representing (NSR) primary sampling units (PSU), we observed unusually high value of 𝑤𝑏𝑠(𝑞) . • We wanted to better understand the cause of this result. 4

Current Population Survey • One of the oldest surveys in the U.S. (in operation since 1942) • Measuring national unemployment rate • Monthly sample of ~72,000 households 5

Primary Sampling Unit • PSU - either a county or group of contiguous counties • Two types of PSU: • Self-representing SR • Non-self-representing NSR 6

CPS Sample Design • Two-stage stratified sampling design for NSR PSUs: • First stage : select one PSU per stratum with probability proportional to size (civilian noninstitutionalized population 16+ = CNP 16+) • Second stage : do systematic sampling within selected PSUs • Systematic sampling for SR PSUs 7

CPS Sample Design • Select PSUs once every 10 years • 852 PSUs selected in first-stage (2010 design): 506 SR and 346 NSR • Approximately 80% of CNP 16+ population in SR PSUs 8

Key Labor Force Estimates • Noninstitutionalized civilian labor force statistics: • Unemployment/employment levels • Unemployment rate • Labor force participation rate 9

Problem • Estimate monthly response rate 𝑞, variance 𝑤𝑏𝑠(𝑞) for CPS households in NSR PSUs . • The sample is at household level: 1 record for each sampled household in each month. • Response 𝑧 𝑗 has binary outcome: 1--response and 0-- nonresponse. • Time period: March 17 – March 18 10

NSR Household Response Rates March 17 – March 18 Household Response Rate 11

Estimated Varia iance for Response an and No Nonresponse Ra Rates Mar ar 17 17 – Mar ar 18 18 • Expect to see 𝒘𝒃𝒔 𝒒 = 𝒘𝒃𝒔(𝟐 − 𝒒) , but they are NOT. • Our chosen variance estimator introduces bias in some way. 12

CPS Vari riance Estimation • Due to CPS sample design, there is no direct variance estimator formula: • Select only one PSU per NSR stratum • Systematic sampling within PSU • Currently use balanced-repeated replication (BRR) method for NSR PSUs . 13

CPS Vari riance Estimation • BRR variance estimator: 𝑆 1 𝑤𝑏𝑠 ෠ ( ෠ 𝑠 − ෠ 𝑍) 2 𝑍 = 𝑆(1 − 𝐿) 2 ෍ 𝑍 𝑠=1 where ෠ 𝑍 𝑠 = the 𝑠 -th replicate estimate of 𝑍 ෠ 𝑍 = the full sample estimate of 𝑍 𝑆 = number of replicates 𝐿 = perturbation factor; 0 ≤ 𝐿 < 1 • BRR requires selecting two PSUs per stratum, but CPS selects only one PSU per stratum  collapse strata to make pseudo-strata. • These pseudo-strata should ideally contain exactly 2 perfectly matched strata . 14

BRR wit ith Pseudo-Strata • Suppose we want to estimate a population total 𝑍 using ෠ ෠ 𝑀 𝑍 = σ ℎ=1 𝑍 ℎ where 𝑀 denotes the number of strata. • Consider the simple case when 𝑀 is even, we estimate the variance of ෠ 𝑍 by combining the 𝑀 strata into 𝐻 groups of two strata each ( 𝑀 = 2𝐻 ). 15

BRR wit ith Pseudo-Strata • Hence, 𝐻 𝐻 ෠ ෠ (෠ 𝑕1 + ෠ 𝑍 = ෍ 𝑍 𝑕 = ෍ 𝑍 𝑍 𝑕2 ) 𝑕=1 𝑕=1 𝑯 𝑯 𝟑 + 𝝉 𝒉𝟑 𝟑 ) 𝑾𝒃𝒔 ෡ 𝑾𝒃𝒔(෡ 𝒁 𝒉𝟐 ) + 𝑾𝒃𝒔(෡ 𝒁 = ෍ 𝒁 𝒉𝟑 ) = ෍ (𝝉 𝒉𝟐 𝒉=𝟐 𝒉=𝟐 16

BRR wit ith Pseudo-Strata • The 𝑠 -th replicate estimate of 𝑍 : 𝐻 ෠ (1 + (1 − 𝐿)𝜀 𝑕𝑠 ) ෠ 𝑕1 + (1 − (1 − 𝐿)𝜀 𝑕𝑠 )෠ 𝑍 𝑠 = ෍ 𝑍 𝑍 𝑕2 𝑕=1 where 𝜀 𝑕𝑠 = 1 if the first stratum in 𝑕 -th group is selected and 𝜀 𝑕𝑠 = − 1 if the second stratum in 𝑕 -th group is selected. • 𝜀 𝑕𝑠 are chosen from entries of a Hadamard matrix. • Rows of a Hadamard matrix are mutually orthogonal: 𝑆 ෍ 𝜀 𝑕𝑠 𝜀 𝑙𝑠 = 0 (∀ 𝑕 ≠ 𝑙) 𝑠=1 17

BRR wit ith Pseudo-Strata 𝐻 ෠ 𝑠 − ෠ 1 − 𝐿 𝜀 𝑕𝑠 ෠ 𝑕1 − ෠ 𝑍 𝑍 = ෍ 𝑍 𝑍 𝑕2 𝑕=1 𝐻 2 1 − 𝐿 2 𝜀 𝑕𝑠 (෠ 𝑠 − ෠ ෠ 𝑕1 − ෠ 𝑍) 2 = ෍ 2 𝑍 𝑍 𝑍 𝑕2 𝑕=1 𝐻 𝐻 1 − 𝐿 2 𝜀 𝑕𝑠 𝜀 𝑙𝑠 ෠ 𝑕1 − ෠ ෠ 𝑙1 − ෠ + ෍ ෍ 𝑍 𝑍 𝑍 𝑍 𝑕2 𝑙2 𝑕=1 𝑙≠𝑕 18

BRR with Pseudo-Strata 𝑆 𝑆 𝐻 1 1 2 𝑍) 2 = (෠ 𝑠 − ෠ ෠ 𝑕1 − ෠ 1 − 𝐿 2 𝑆(1 − 𝐿) 2 ෍ 𝑍 𝑆(1 − 𝐿) 2 ෍ ෍ 𝑍 𝑍 𝑕2 𝑠=1 𝑠=1 𝑕=1 𝐻 𝐻 𝑆 1 1 − 𝐿 2 ෠ 𝑕1 − ෠ ෠ 𝑙1 − ෠ + 𝑆(1 − 𝐿) 2 ෍ ෍ 𝑍 𝑍 𝑍 𝑍 𝑙2 ෍ 𝜀 𝑕𝑠 𝜀 𝑙𝑠 𝑕2 𝑕=1 𝑙≠𝑕 𝑠=1 • Therefore, 𝐻 𝐻 2 +෠ 2 −2෠ 𝑕2 ) 2 = ෍ 𝑤𝑏𝑠 ෠ (෠ 𝑕1 − ෠ (෠ 𝑕1 ෠ 𝑍 = ෍ 𝑍 𝑍 𝑍 𝑍 𝑍 𝑍 𝑕2 ) 𝑕1 𝑕2 𝑕=1 𝑕=1 19

Bia ias in in BRR wit ith Pseudo-Strata • Taking expectation: 𝐻 2 +෠ 2 −2෠ 2 + 𝑊𝑏𝑠 ෠ 2 − 2𝜈 𝑕1 𝜈 𝑕2 ( ෠ 𝑕1 ෠ = 𝑊𝑏𝑠 ෠ 𝐹 ෍ 𝑍 𝑍 𝑍 𝑍 𝑕2 ) 𝑍 𝑕1 + 𝜈 𝑕1 𝑍 𝑕2 + 𝜈 𝑕2 𝑕1 𝑕2 𝑕=1 𝐻 𝐻 2 + 𝜏 2 ) + ෍ (𝜈 𝑕1 − 𝜈 𝑕2 ) 2 = ෍ (𝜏 𝑕1 𝑕2 𝑕=1 𝑕=1 = 𝑊𝑏𝑠(෠ 𝑍) + 𝐶𝑗𝑏𝑡 2 2 = Var{෠ 𝑕ℎ } and 𝜈 𝑕ℎ = 𝐹{෠ where 𝜏 𝑍 𝑍 𝑕ℎ } . 𝑕ℎ • Bias squared term is positive and would ADD to variance estimate. • Bias squared term would be zero if the pair of PSUs in each group were perfectly matched. 20

How are Strata Coll llapsed ? • In CPS, the objective function is a function of: • Unemployment • Civilian labor force • Children 0-17 at or below 200% poverty level 21

Sim imulation Overview • Use one month CPS data (Mar 18) which has pseudo-strata information. • For each household, generate 𝑧 𝑗 responses iid from Bernoulli distribution with various 𝑞 = 0.03, 0.06, … , 0.99 . • For each 𝑞: • Run 5,000 sims. • Compute true variance and BRR variance. 𝐻 (𝜈 𝑕1 − 𝜈 𝑕2 ) 2 • Compute bias squared term σ 𝑕=1 • Compare true variance with BRR variance after adjusting for bias. 22

Simulation Computation • Total number of households: ෡ 𝑜 𝑂 = σ 𝑗=1 𝑥 𝑗 • Full sample estimated response count: ෠ 𝑜 𝑍 = σ 𝑗=1 𝑥 𝑗 𝑧 𝑗 • Replicate 𝑠 estimated response count: ෠ 𝑜 𝑠 = σ 𝑗=1 𝑍 𝑥 𝑗 𝑧 𝑗 𝑔 𝑗𝑠 where 𝑔 𝑗𝑠 is either 1.5 or 0.5. 23

Simulation Computation 4 • BRR variance of ෠ 𝑍: 𝐶𝑆𝑆𝑊𝑏𝑠 ෠ 160 (෠ 𝑠 − ෠ 𝑍) 2 160 σ 𝑠=1 𝑍 = 𝑍 • BRR variance of response rate 𝑞 : 2 1 𝐶𝑆𝑆𝑊𝑏𝑠 ෠ 𝐶𝑆𝑆𝑊𝑏𝑠 𝑞 = 𝑍 ෡ 𝑂 assuming 𝑂 = ෡ 𝑂 is fixed from outside knowledge. • 𝜈 𝑕ℎ = 𝑞 × 𝑂 𝑕ℎ 24

BRR Variance vs. Tru rue Variance As 𝒒 gets close to 1, BRR variance estimate is far BRRVar different from true variance. TrueVar 25

BRR Variance vs. Tru rue Variance Not all of bias can be explained due to: BRRVar • Strata collapsed based on different set of covariates • Use AHS MOS instead of CPS • AHS MOS not currently updated (from 2010 design) BRRVar - BiasSq TrueVar 26

Summary ry • For NSR component: • CPS collapses strata to make pseudo-strata. • There is no perfect matching of strata  bias in variance estimator. • Bias gets significantly large when 𝑞 gets close to 1. • Quick fix is to use 𝑤𝑏𝑠(1 − 𝑞) for large 𝑞 . • CPS is designed for civilian labor force statistics. Expect more bias when estimating variance of other statistics. 27

Questions? Thank You! khoa.dong@census.gov 28

References 1.David Judkins (1990). “Fay’s method for variance estimation.” Journal of Official Statistics, Vol 6, No. 3, 1990 2. Philip J. McCarthy (1966). “Replication: An Approach to the Analysis of Data from Complex Surveys.” Vital and Health Statistics Series 2 No. 14 3. Robert E. Fay (1984). “Some Properties of Estimates of Variances Based on Replication Methods.” 4. Philip J. McCarthy (1969). “Pseudo - Replication: Half Samples.” Review of the International Statistical Institute, Vol. 37, No. 3, pp. 239-264 5. Yang Cheng (2012). “Overview of Current Population Survey Methodology.” Internal Report. 6.Wolter, K.M. (2008). Introduction to Variance Estimation, New York: Spring-Verlag. 29