Quantitative Aspects of Screening Michael O’Reilly, MD, MPH Technical Advisor Field Epidemiology Training Program - Thailand
Objectives • Understand the concepts of sensitivity, specificity, and predictive values • Be able to calculate the testing characteristics above • Know what factors drive predictive values
Questions • How do we evaluate the ability of a test to “do what it’s supposed to do?” • How well does it identify those who have disease • How well does it exclude those who don’t have disease
Sensitivity Sensitivity What is the definition?
Disease - + + a b (true +) (false +) Test c d - (false -) (true -)
Sensitivity Sensitivity The likelihood of a person with disease having a positive test: _a_ a + c
Disease - + + a b (true +) (false +) Test c d - (false -) (true -)
Sensitivity Sensitivity This is a vertical property I mmutable Concerned only with individuals who have disease
102 48 - Influenza 117 33 + Rapid + - Test
Sens = ? 102 48 - Influenza 117 33 + Rapid + - Test
Influenza + - Sens = 117 48 + 117/150 Rapid = 78% Test 33 102 -
Sensitivity Sensitivity I s it affected by prevalence? What happens if we double the number of individuals with flu in our study?
Influenza + - Sens = 234 48 + 234/300 Rapid = 78% Test 66 102 -
Specificity Specificity What is the definition?
Specificity Specificity The likelihood of a person without disease having a negative test: _d_ d + b
Disease - + + a b (true +) (false +) Test c d - (false -) (true -)
Specificity Specificity This is a vertical property I mmutable Concerned only with individuals who don’t have disease
Spec = ? 102 48 - Influenza 117 33 + Rapid + - Test
Influenza + - Spec = 117 48 + 102/150 Rapid = 67% Test 33 102 -
Relationship of Sensitivity and Relationship of Sensitivity and Specificity Specificity How are these two characteristics related? Under ordinary circumstances, what happens to specificity if you increase sensitivity?
Sensitivity & Specificity: Establishing a Threshold Value * * * * * * * * * * * * * * * * * * * * * * * * CK MB band * * * 8.0 * * * * * * * * * * * * * * * * * * * * * * * * * * * * MI patients non MI patients
Sensitivity & Specificity are Inversely Related sensitivity 1 - specificity
Sensitivity & Specificity: Establishing a Threshold Value * * * * * * * * * * * * * * * * * * * * * * * * CK MB band * * * 8.0 * * * * * * * * * * * * * * * * * * * * * * * * * * * * MI patients non MI patients
Maximizing Sensitivity * * * * * * * * * * * * * * * * * * * * * * * * CK MB band * * * * * * * * * * * 4.0 * * * * * * * * * * * * * * * * * * * * MI patients non MI patients
Gains in Sensitivity mean losses in Specificity sensitivity 1 - specificity
Sensitivity & Specificity: Changing Threshold Values * * * * * * * * * * * * * * * * * * * * * * * * CK MB band 12.0 * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * MI patients non MI patients
Gains in Specificity mean losses in Sensitivity sensitivity 1 - specificity
Positive Predictive Value Positive Predictive Value What is the definition?
Positive Predictive Value Positive Predictive Value The likelihood of a person with positive test really having disease: _a_ a + b
Disease - + + a b (true +) (false +) Test c d - (false -) (true -)
PPV = ? 102 48 - Influenza 117 33 + Rapid + - Test
Influenza + - PPV =117/165 117 48 + =71% Rapid Test 33 102 -
Positive Predictive Value Positive Predictive Value I s it affected by prevalence? What happens if we double the number of individuals with flu in our study?
Influenza + - PPV = 234/282 234 48 + = 83% Rapid Test 66 102 -
Positive Predictive Value Positive Predictive Value This is a horizontal property Mutable Concerned both with individuals who have and don’t have disease
Positive Predictive Value: Example Positive Predictive Value: Example You are offered a PSA test with sensitivity 90% and specificity 80% I n a cancer clinic population of 1,000 prevalence is 75% . What is PPV? I n an asymptomatic population of 1,000 prevalence is .5% . What is the PPV? Where do you use the test?
Prostate Cancer + - a b + PSA Test c d - Total = 1,000
Prostate Cancer + - Prevalence: 75% a b + 1,000 x .75 = 750 PSA Test a + c = 750 c d - 750 Total = 1,000
Prostate Cancer + - a + c = 750 a b + d + b = PSA 1,000 - 750 = 250 Test c d - 750 250 Total = 1,000
Prostate Cancer + - Sensitivity = .90 675 b a = 750 x .90 + =675 PSA Test [c = 750 – 675] 75 d - 750 250 Total = 1,000
Prostate Cancer + - Specificity = .80 675 50 d = 250 x .80 + =200 PSA Test [b = 250 – 200] 75 200 - 750 250 Total = 1,000
Prostate Cancer + - PPV = a/a+b 675 50 a + b = 725 + PPV = 675/725 PSA = 93% Test 75 200 - 750 250 Total = 1,000
Prostate Cancer + - 4.5 199 a + b = 203.5 + PPV = 2.2% PSA Test 0.5 796 - 5 995 Total = 1,000
Positive Predictive Value: Positive Predictive Value: Comments Comments Affected greatly by prevalence One other factor has a great impact: is it sensitivity or specificity?
PPV: I nfluence of sensitivity and PPV: I nfluence of sensitivity and specificity specificity I n a general population of 1,000 prevalence of prostate cancer is .5% PSA Laboratories, inc. has heard your criticisms and has made 2 new PSA tests: Super Sensitive PSA [sens= .98; spec= .80] • Super Specific PSA [sens= .80; spec= .98] • Which test has the higher PPV?
Prostate Cancer + - Super Sensitive 4.9 199 + Sens .98 Spec .80 PSA Test 0.1 796 PPV = 2.4% - 5 995 Total = 1,000
Prostate Cancer + - Super Specific 4 19.9 + Sens .80 Spec .98 PSA Test 1 975.1 PPV = 16.7% - 5 995 Total = 1,000
PPV: Summary PPV: Summary Driven by specificity and prevalence Most powerful in settings of moderate disease prevalence Can describe screening results more powerfully than sensitivity or specificity
Disease highly prevalent - + + b A (false +) (true +) Test d C - (true -) (false -)
Test highly specific - + b + a (false +) (true +) Test c D - (false -) (true -)
Testing Characteristics Testing Characteristics What property is most important for a pregnancy test? What property is most important for a confirmatory colon cancer test?
Recommend
More recommend
