hypothesis testing
play

Hypothesis Testing Part III: Pretest Bias James J. Heckman - PowerPoint PPT Presentation

Oct 26, 2022 •478 likes •690 views

Hypothesis Testing Part III: Pretest Bias James J. Heckman University of Chicago Based on T.A. Bancroft, The Annals of Mathematical Statistics , Vol. 15, No. 2. (June, 1944), pp. 190-204. Econ 312 Spring 2019 1 A

  1. Hypothesis Testing Part III: Pretest Bias James J. Heckman University of Chicago Based on T.A. Bancroft, The Annals of Mathematical Statistics , Vol. 15, No. 2. (June, 1944), pp. 190-204. Econ 312 Spring 2019 1

  2. � � � � � � � A common procedure almost universal in economics is to select a model on the basis of a preliminary test. Example: = � � � + � � � ( � � | � � ) = 0 = [ � 1 � � � 2 � ] = ( � 1 � � 2 ) Include � 2 if a “ � ” statistic on ˆ � 2 is big enough. 2

  3. ˆ ˆ ˜ � 1 ,pretest = � 1 1 ( � � 2 � � ) + � 1 1 ( � � 2 � � ) ˆ � 1 comes from unrestricted model; ˜ � 1 comes from model deleting � 2 . 3

  4. For a fixed size test, say � , � % of time we reject when null � 0 : � 2 = 0 is true. � Estimator is inconsistent unless we change � with sample size (e.g. � � � 0 ). Bayesians show it’s inadmissible under quadratic loss. Examples of Bias: 4

  5. Model : Y i = X i β +  i ; i = 1,...,N ρ X 1 0, 1 X ≡ ∼ N ; iid ρ X 2 1  ∼ N  0, σ 2  ; iid Pre-Test Estimator Density Function Pre-Test Estimator Density Function β 1 density without drop of β 2 β 2 density without drop of β 2 β 1 N = 50 β 1 N = 50 β 1 density with drop of β 2 β 2 density with drop of β 2 1.2 1.2 Probability Density Function of estimated Probability Density Function of estimated 1 1 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0 0 -2 -1 0 1 2 3 4 -2 -1 0 1 2 3 4 β 1 , ρ =0.9 σ =10 β 2 , ρ =0.9 σ =10 Distribution of β 1 Distribution of β 2 N = 50, ρ = 0.9; σ 2 = 10 N = 50, ρ = 0.9; σ 2 = 10 5

  6. Pre-Test Estimator Density Function Pre-Test Estimator Density Function β 1 density without drop of β 2 β 2 density without drop of β 2 β 1 N = 50 β 1 N = 50 β 1 density with drop of β 2 β 2 density with drop of β 2 1.2 1.2 Probability Density Function of estimated Probability Density Function of estimated 1 1 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0 0 -2 -1 0 1 2 3 4 -2 -1 0 1 2 3 4 β 1 , ρ =0.75 σ =10 β 2 , ρ =0.75 σ =10 Distribution of β 1 Distribution of β 2 N = 50, ρ = 0.75; σ 2 = 10 N = 50, ρ = 0.75; σ 2 = 10 6

  7. Pre-Test Estimator Density Function Pre-Test Estimator Density Function β 1 density without drop of β 2 β 2 density without drop of β 2 β 1 N = 50 β 1 N = 50 β 1 density with drop of β 2 β 2 density with drop of β 2 1.2 1.2 Probability Density Function of estimated Probability Density Function of estimated 1 1 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0 0 -2 -1 0 1 2 3 4 -2 -1 0 1 2 3 4 β 1 , ρ =0.5 σ =10 β 2 , ρ =0.5 σ =10 Distribution of β 1 Distribution of β 2 N = 50, ρ = 0.5; σ 2 = 10 N = 50, ρ = 0.5; σ 2 = 10 7

  8. Pre-Test Estimator Density Function Pre-Test Estimator Density Function β 1 density without drop of β 2 β 2 density without drop of β 2 β 1 N = 50 β 1 N = 50 β 1 density with drop of β 2 β 2 density with drop of β 2 1.2 1.2 Probability Density Function of estimated Probability Density Function of estimated 1 1 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0 0 -2 -1 0 1 2 3 4 -2 -1 0 1 2 3 4 β 1 , ρ =0.25 σ =10 β 2 , ρ =0.25 σ =10 Distribution of β 1 Distribution of β 2 N = 50, ρ = 0.25; σ 2 = 10 N = 50, ρ = 0.25; σ 2 = 10 8

  9. Pre-Test Estimator Density Function Pre-Test Estimator Density Function β 1 density without drop of β 2 β 2 density without drop of β 2 β 1 N = 50 β 1 N = 50 β 1 density with drop of β 2 β 2 density with drop of β 2 1.2 1.2 Probability Density Function of estimated Probability Density Function of estimated 1 1 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0 0 -2 -1 0 1 2 3 4 -2 -1 0 1 2 3 4 β 1 , ρ =0.9 σ =5 β 2 , ρ =0.9 σ =5 Distribution of β 1 Distribution of β 2 N = 50, ρ = 0.9; σ 2 = 5 N = 50, ρ = 0.9; σ 2 = 5 9

  10. Pre-Test Estimator Density Function Pre-Test Estimator Density Function β 1 density without drop of β 2 β 1 density without drop of β 2 β 1 N = 50 β 1 N = 50 β 1 density with drop of β 2 β 1 density with drop of β 2 1.2 1.2 Probability Density Function of estimated Probability Density Function of estimated 1 1 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0 0 -2 -1 0 1 2 3 4 -2 -1 0 1 2 3 4 β 1 , ρ =0.75 σ =5 β 1 , ρ =0.75 σ =5 Distribution of β 1 Distribution of β 2 N = 50, ρ = 0.75; σ 2 = 5 N = 50, ρ = 0.75; σ 2 = 5 10

  11. Pre-Test Estimator Density Function Pre-Test Estimator Density Function β 1 density without drop of β 2 β 2 density without drop of β 2 β 1 N = 50 β 1 N = 50 β 1 density with drop of β 2 β 2 density with drop of β 2 1.2 1.2 Probability Density Function of estimated Probability Density Function of estimated 1 1 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0 0 -2 -1 0 1 2 3 4 -2 -1 0 1 2 3 4 β 1 , ρ =0.5 σ =5 β 2 , ρ =0.5 σ =5 Distribution of β 1 Distribution of β 2 N = 50, ρ = 0.5; σ 2 = 5 N = 50, ρ = 0.5; σ 2 = 5 11

  12. Pre-Test Estimator Density Function Pre-Test Estimator Density Function β 1 density without drop of β 2 β 2 density without drop of β 2 β 1 N = 50 β 1 N = 50 β 1 density with drop of β 2 β 2 density with drop of β 2 1.2 1.2 Probability Density Function of estimated Probability Density Function of estimated 1 1 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0 0 -2 -1 0 1 2 3 4 -2 -1 0 1 2 3 4 β 1 , ρ =0.25 σ =5 β 2 , ρ =0.25 σ =5 Distribution of β 1 Distribution of β 2 N = 50, ρ = 0.25; σ 2 = 5 N = 50, ρ = 0.25; σ 2 = 5 12

  13. Pre-Test Estimator Density Function Pre-Test Estimator Density Function β 1 density without drop of β 2 β 2 density without drop of β 2 β 1 N = 50 β 1 N = 50 β 1 density with drop of β 2 β 2 density with drop of β 2 1.2 1.2 Probability Density Function of estimated Probability Density Function of estimated 1 1 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0 0 -2 -1 0 1 2 3 4 -2 -1 0 1 2 3 4 β 1 , ρ =0.9 σ =2 β 2 , ρ =0.9 σ =2 Distribution of β 1 Distribution of β 2 N = 50, ρ = 0.9; σ 2 = 2 N = 50, ρ = 0.9; σ 2 = 2 13

  14. Pre-Test Estimator Density Function Pre-Test Estimator Density Function β 1 density without drop of β 2 β 2 density without drop of β 2 β 1 N = 50 β 1 N = 50 β 1 density with drop of β 2 β 2 density with drop of β 2 1.2 1.2 Probability Density Function of estimated Probability Density Function of estimated 1 1 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0 0 -2 -1 0 1 2 3 4 -2 -1 0 1 2 3 4 β 1 , ρ =0.75 σ =2 β 2 , ρ =0.75 σ =2 Distribution of β 1 Distribution of β 2 N = 50, ρ = 0.75; σ 2 = 2 N = 50, ρ = 0.75; σ 2 = 2 14

  15. Pre-Test Estimator Density Function Pre-Test Estimator Density Function β 2 density without drop of β 2 β 2 density without drop of β 2 β 1 N = 50 β 1 N = 50 β 2 density with drop of β 2 β 2 density with drop of β 2 1.2 1.2 Probability Density Function of estimated Probability Density Function of estimated 1 1 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0 0 -2 -1 0 1 2 3 4 -2 -1 0 1 2 3 4 β 2 , ρ =0.5 σ =2 β 2 , ρ =0.5 σ =2 Distribution of β 1 Distribution of β 2 N = 50, ρ = 0.5; σ 2 = 2 N = 50, ρ = 0.5; σ 2 = 2 15

  16. Pre-Test Estimator Density Function Pre-Test Estimator Density Function β 1 density without drop of β 2 β 2 density without drop of β 2 β 1 N = 50 β 1 N = 50 β 1 density with drop of β 2 β 2 density with drop of β 2 1.2 1.2 Probability Density Function of estimated Probability Density Function of estimated 1 1 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0 0 -2 -1 0 1 2 3 4 -2 -1 0 1 2 3 4 β 1 , ρ =0.25 σ =2 β 2 , ρ =0.25 σ =2 Distribution of β 1 Distribution of β 2 N = 50, ρ = 0.25; σ 2 = 2 N = 50, ρ = 0.25; σ 2 = 2 16

  17. Pre-Test Estimator Density Function Pre-Test Estimator Density Function β 1 density without drop of β 2 β 2 density without drop of β 2 β 1 N = 50 β 1 N = 50 β 1 density with drop of β 2 β 2 density with drop of β 2 1.2 1.2 Probability Density Function of estimated Probability Density Function of estimated 1 1 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0 0 -2 -1 0 1 2 3 4 -2 -1 0 1 2 3 4 β 1 , ρ =0.9 σ =1 β 2 , ρ =0.9 σ =1 Distribution of β 1 Distribution of β 2 N = 50, ρ = 0.9; σ 2 = 1 N = 50, ρ = 0.9; σ 2 = 1 17

  18. Pre-Test Estimator Density Function Pre-Test Estimator Density Function β 1 density without drop of β 2 β 2 density without drop of β 2 β 1 N = 50 β 1 N = 50 β 1 density with drop of β 2 β 2 density with drop of β 2 1.2 1.2 Probability Density Function of estimated Probability Density Function of estimated 1 1 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0 0 -2 -1 0 1 2 3 4 -2 -1 0 1 2 3 4 β 1 , ρ =0.75 σ =1 β 2 , ρ =0.75 σ =1 Distribution of β 1 Distribution of β 2 N = 50, ρ = 0.75; σ 2 = 1 N = 50, ρ = 0.75; σ 2 = 1 18

  19. Pre-Test Estimator Density Function Pre-Test Estimator Density Function β 1 density without drop of β 2 β 2 density without drop of β 2 β 1 N = 50 β 1 N = 50 β 1 density with drop of β 2 β 2 density with drop of β 2 1.2 1.2 Probability Density Function of estimated Probability Density Function of estimated 1 1 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0 0 -2 -1 0 1 2 3 4 -2 -1 0 1 2 3 4 β 1 , ρ =0.5 σ =1 β 2 , ρ =0.5 σ =1 Distribution of β 1 Distribution of β 2 N = 50, ρ = 0.5; σ 2 = 1 N = 50, ρ = 0.5; σ 2 = 1 19

Recommend

Gov 2000: 6. Hypothesis Testing Matthew Blackwell October 11, 2016 1 / 55 1. Hypothesis

Gov 2000: 6. Hypothesis Testing Matthew Blackwell October 11, 2016 1 / 55 1. Hypothesis

Gov 2000: 6. Hypothesis Testing Matthew Blackwell October 11, 2016 1 / 55 1. Hypothesis Testing Examples 2. Hypothesis Test Nomenclature 3. Conducting Hypothesis Tests 4. p-values 5. Power Analyses 6. Exact Inference* 7. Wrap up 2 / 55

929 views • 63 slides
Chapter 6 Hypothesis Testing What is Hypothesis Testing? the use of statistical

Chapter 6 Hypothesis Testing What is Hypothesis Testing? the use of statistical

Chapter 6 Hypothesis Testing What is Hypothesis Testing? the use of statistical procedures to answer research questions Typical research question (generic): For hypothesis testing, research questions are statements: This is

770 views • 50 slides
Chapter 6 Hypothesis Testing What is Hypothesis Testing? the use of statistical

Chapter 6 Hypothesis Testing What is Hypothesis Testing? the use of statistical

Chapter 6 Hypothesis Testing What is Hypothesis Testing? the use of statistical procedures to answer research questions Typical research question (generic): For hypothesis testing, research questions are statements: This is

243 views • 22 slides
Testing 6.1 Specification testing Michel Bierlaire A short reminder on hypothesis testing

Testing 6.1 Specification testing Michel Bierlaire A short reminder on hypothesis testing

Testing 6.1 Specification testing Michel Bierlaire A short reminder on hypothesis testing Hypothesis testing is a method to contradict a theoretical assumption using data. In his seminal book on design of experiments, Fisher (1937) uses

526 views • 3 slides
Statistics'and' Hypothesis'Testing

Statistics'and' Hypothesis'Testing

Statistics'and' Hypothesis'Testing NENS230:DataAnalysisfortheBiosciencesusingMATLAB EddyAlbarran November3,2015 AnalysisMethodology Data Exploratory Hypothesis DataAnalysis Testing

985 views • 61 slides
Hypothesis Testing for a Proportion August 21, 2019 August 21, 2019 1 / 64 Hypothesis Testing

Hypothesis Testing for a Proportion August 21, 2019 August 21, 2019 1 / 64 Hypothesis Testing

Hypothesis Testing for a Proportion August 21, 2019 August 21, 2019 1 / 64 Hypothesis Testing Framework Suppose were interested in examining how people perform on a multiple choice question related to world health. We might like to

1.37k views • 64 slides
STAT 113 Hypothesis Testing I Colin Reimer Dawson Oberlin College October 5, 2017 1 / 17

STAT 113 Hypothesis Testing I Colin Reimer Dawson Oberlin College October 5, 2017 1 / 17

Hypothesis Testing Null Hypothesis Testing Formulating Statistical Hypotheses Preview: Quantifying Evidence STAT 113 Hypothesis Testing I Colin Reimer Dawson Oberlin College October 5, 2017 1 / 17 Hypothesis Testing Null Hypothesis

321 views • 17 slides
Hypothesis testing get data that differ from the null hypothesis. If the data would be quite

Hypothesis testing get data that differ from the null hypothesis. If the data would be quite

Hypothesis testing asks how unusual it is to Hypothesis testing get data that differ from the null hypothesis. If the data would be quite unlikely under H 0 , we reject H 0 . So we need to know how good the sample is, and how likely it is that

334 views • 14 slides
STAT 215 Hypothesis Testing I Colin Reimer Dawson Oberlin College September 7, 2017 1 / 14

STAT 215 Hypothesis Testing I Colin Reimer Dawson Oberlin College September 7, 2017 1 / 14

Inference Goals Hypothesis Testing Philosophy of Null Hypothesis Testing Formulating Statistical Hypotheses STAT 215 Hypothesis Testing I Colin Reimer Dawson Oberlin College September 7, 2017 1 / 14 Inference Goals Hypothesis Testing

251 views • 14 slides
Sequential techniques for Hypothesis testing & Change detection George V. Moustakides

Sequential techniques for Hypothesis testing & Change detection George V. Moustakides

Sequential techniques for Hypothesis testing & Change detection George V. Moustakides University of Patras Outline Sequential hypothesis testing The Sequential Probability Ratio Test (SPRT) for optimum hypothesis testing

393 views • 27 slides
Hypothesis Testing Saravanan Vijayakumaran sarva@ee.iitb.ac.in Department of Electrical

Hypothesis Testing Saravanan Vijayakumaran sarva@ee.iitb.ac.in Department of Electrical

Hypothesis Testing Saravanan Vijayakumaran sarva@ee.iitb.ac.in Department of Electrical Engineering Indian Institute of Technology Bombay August 22, 2012 1 / 23 Basics of Hypothesis Testing What is a Hypothesis? One situation among a set of

501 views • 23 slides
Testing Specification testing Michel Bierlaire Introduction to choice models Differences from

Testing Specification testing Michel Bierlaire Introduction to choice models Differences from

Testing Specification testing Michel Bierlaire Introduction to choice models Differences from classical hypothesis testing Classical hypothesis testing: example Null hypothesis ( H 0 ) A simple hypothesis contradicting a theoretical

516 views • 33 slides
Hypothesis testing Hypothesis tests are most often used to make a statement about superiority,

Hypothesis testing Hypothesis tests are most often used to make a statement about superiority,

Hypothesis testing Hypothesis tests are most often used to make a statement about superiority, inferiority or equality of treatments. Confidence intervals do contain this information also, but the conclusion is a bit more difficult to extract.

242 views • 13 slides
Hypothesis Testing Mark Lunt Centre for Epidemiology Versus Arthritis University of Manchester

Hypothesis Testing Mark Lunt Centre for Epidemiology Versus Arthritis University of Manchester

Formulating a hypothesis test Interpreting a hypothesis test Common types of hypothesis test Power calculations Hypothesis tests and confidence intervals Hypothesis Testing Mark Lunt Centre for Epidemiology Versus Arthritis University of

498 views • 48 slides
Hypothesis Testing and statistical preliminaries Stony Brook University CSE545, Spring 2019

Hypothesis Testing and statistical preliminaries Stony Brook University CSE545, Spring 2019

Hypothesis Testing and statistical preliminaries Stony Brook University CSE545, Spring 2019 Hypothesis Testing: Random Variables Distributions Hypothesis Testing Framework Comparing Variables: Simple Linear Regression,

1.37k views • 105 slides
Hypothesis testing DS GA 1002 Statistical and Mathematical Models

Hypothesis testing DS GA 1002 Statistical and Mathematical Models

Hypothesis testing DS GA 1002 Statistical and Mathematical Models http://www.cims.nyu.edu/~cfgranda/pages/DSGA1002_fall15 Carlos Fernandez-Granda Example In a medical study 10% of women and 12.5% of men suffer from heart disease Hypothesis:

1.18k views • 81 slides
Hypothesis tests with binomial example STAT 587 (Engineering) Iowa State University October 2,

Hypothesis tests with binomial example STAT 587 (Engineering) Iowa State University October 2,

Hypothesis tests with binomial example STAT 587 (Engineering) Iowa State University October 2, 2020 Statistical hypothesis testing Statistical hypothesis testing A hypothesis test consists of two hypotheses, null hypothesis ( H 0 ) and an

312 views • 14 slides
15-388/688 - Practical Data Science: Hypothesis testing and experimental design J. Zico Kolter

15-388/688 - Practical Data Science: Hypothesis testing and experimental design J. Zico Kolter

15-388/688 - Practical Data Science: Hypothesis testing and experimental design J. Zico Kolter Carnegie Mellon University Fall 2019 1 Outline Motivation Background: sample statistics and central limit theorem Basic hypothesis testing

654 views • 27 slides
Hypothesis Testing Part I James J. Heckman University of Chicago Econ 312, Spring 2019 Heckman

Hypothesis Testing Part I James J. Heckman University of Chicago Econ 312, Spring 2019 Heckman

Hypothesis Testing Part I James J. Heckman University of Chicago Econ 312, Spring 2019 Heckman Hypothesis Testing Part I 1. A Brief Review of Hypothesis Testing and Its Uses Common Phrase: Chicago Economics test Models What are Valid

1.46k views • 97 slides
Introduction I Introduction I Introduction II Introduction II Statistical inference

Introduction I Introduction I Introduction II Introduction II Statistical inference

Outline: Chapter 8 Hypothesis Testing Outline: Chapter 8 Hypothesis Testing 2 Introduction I d i Hypothesis Testing 204312 PROBABILITY AND 204312 PROBABILITY AND Significance Testing (8.1 Y&G) RANDOM PROCESSES FOR Binary

348 views • 17 slides
Hypothesis Testing with An Important New . . . Interval Data: Case of Case of Probabilistic . .

Hypothesis Testing with An Important New . . . Interval Data: Case of Case of Probabilistic . .

Hypothesis Testing: A . . . Traditional Statistical . . . How to Describe . . . Types of Uncertainty: . . . Hypothesis Testing with An Important New . . . Interval Data: Case of Case of Probabilistic . . . Example: Car Testing . . .

208 views • 18 slides
t -tests STAT 587 (Engineering) Iowa State University October 2, 2020 Statistical hypothesis

t -tests STAT 587 (Engineering) Iowa State University October 2, 2020 Statistical hypothesis

t -tests STAT 587 (Engineering) Iowa State University October 2, 2020 Statistical hypothesis testing Statistical hypothesis testing A hypothesis test consists of two hypotheses: null hypothesis ( H 0 ) and an alternative hypothesis ( H A )

325 views • 10 slides
Lecture 4: Hypothesis Testing Ani Manichaikul amanicha@jhsph.edu 20 April 2007 1 / 69 Steps of

Lecture 4: Hypothesis Testing Ani Manichaikul amanicha@jhsph.edu 20 April 2007 1 / 69 Steps of

Lecture 4: Hypothesis Testing Ani Manichaikul amanicha@jhsph.edu 20 April 2007 1 / 69 Steps of Hypothesis Testing Define the null hypothesis, H 0 Define the alternative hypothesis, H a , where H a is usually of the form not H 0 Define

975 views • 69 slides
Experimental Design & Evaluation 11. Controlled Experiment SunyoungKim,PhD

Experimental Design & Evaluation 11. Controlled Experiment SunyoungKim,PhD

Experimental Design & Evaluation 11. Controlled Experiment SunyoungKim,PhD Todays agenda Hypothesis testing Threats Experimental Biases Hypothesis Testing Hypothesis testing How to prove a hypotheses in

547 views • 34 slides

More recommend