a test statistic for weighted runs
play

A Test Statistic for Weighted Runs Frederik Beaujean, Allen Caldwell - PowerPoint PPT Presentation

Jun 29, 2023 •133 likes •293 views

A Test Statistic for Weighted Runs Frederik Beaujean, Allen Caldwell http://arxiv.org/abs/1005.3233v2 COMPSTAT 2010 Paris, 23.8.2010 Motivating example Suppose: y i Measurements with Gaussian uncertainty Standard Model (SM)

  1. A Test Statistic for Weighted Runs Frederik Beaujean, Allen Caldwell http://arxiv.org/abs/1005.3233v2 COMPSTAT 2010 Paris, 23.8.2010

  2. Motivating example Suppose: y i Measurements with Gaussian uncertainty ● Standard Model (SM) background is quadratic ● New physics (NP) predicts signal peak ● 23.08.2010 Frederik Beaujean #2

  3. Goodness of Fit: standard approach Test statistic: Any scalar function of data, T(D) ● Interpret: large T(D) = poor model ● ∝ ∏ exp { −  y i − f  x i ∣ } = exp { Example:   2 2 } 2 − P  D ∣ Prob. density of the data ● 2 2  i 2  D  T  D ≡ Familiar choice ● 23.08.2010 Frederik Beaujean #3

  4. p-value Def: p ≡ P  T  T  D  p T(D) Assuming the model and before data is taken: ● p uniform in [0,1] Critical values: p  0.05,0.01 ⇒ reject model ● Warning: p-value not the P . that the model is true ● Example: p SM = 10%, p NP = 37% ⇒ both OK 23.08.2010 Frederik Beaujean #4

  5. Runs Most statistics disrespect order ● of data, information wasted Human brain good for simple ● problems Example: N=25 datapoints ● Each Gaussian with mean = 0 ● and variance = 1 Can we combine information about order and magnitude of deviation ? 23.08.2010 Frederik Beaujean #5

  6. Runs statistic Proposal: Split data into runs ● Each run has a weight ● Gaussian case: T est statistic: largest weight of ● any run p-value becomes ● 23.08.2010 Frederik Beaujean #6

  7. Runs distribution Gaussian case: Distribution of T exactly ● calculated for any N (non- parametric) Requires sum over integer ● partitions N = 25 23.08.2010 Frederik Beaujean #7

  8. Power 5% level New physics contribution: ● T up to 35% more ● powerful than classic in detecting departures of type y(x) Lorentz peak with amplitude A 23.08.2010 Frederik Beaujean #8

  9. Conclusions choose statistic with specific alternative models in mind ● Runs statistic T excellent for “bump hunting” ● FINIS FINIS 23.08.2010 Frederik Beaujean #9

  10. Backup 23.08.2010 Frederik Beaujean #10

  11. Exact runs distribution I 23.08.2010 Frederik Beaujean #11

  12. Exact runs distribution II 23.08.2010 Frederik Beaujean #12

  13. Exact runs distribution III 23.08.2010 Frederik Beaujean #13

  14. Computational complexity: Integer partitions 23.08.2010 Frederik Beaujean #14

  15. Goodness of Fit: Bayesian approach Model selection: Need explicit alternatives M 1 , M 2 P  M 1 ∣ D  P  M 2 ∣ D = P  M 1  P  M 2 × P  D ∣ M 1  ● P  D ∣ M 2  Posterior odds ● Bayes factor: (very) sensitive to parameter range ● P  D ∣ M 1 = ∫ p  D ∣  p 0   d   Occam's razor built in ● Example: P  SM ∣ D  P  NP ∣ D  = P  SM  P  NP  × 61.7 Six (NP) vs three (SM) parameters ● 23.08.2010 Frederik Beaujean #15

Recommend

- chisq.test(x, y) runs <- 1000 rbeta(runs, shape1, shape2) runs <- 1000 experiment_1

- chisq.test(x, y) runs <- 1000 rbeta(runs, shape1, shape2) runs <- 1000 experiment_1

- chisq.test(x, y) runs <- 1000 rbeta(runs, shape1, shape2) runs <- 1000 experiment_1 <- rbeta(runs, shape1, shape2) experiment_2 <- rbeta(runs, shape1, shape2) runs <- 1000 experiment_1 <- rdirichlet(runs, alpha = c(a, b,

482 views • 20 slides
Weighted graphs 2 Weighted graphs So far we have only considered weighted graphs with

Weighted graphs 2 Weighted graphs So far we have only considered weighted graphs with

1 Weighted graphs 2 Weighted graphs So far we have only considered weighted graphs with weights > 0 (Dijkstra is a super-star here) Now we will consider graphs with any integer edge weight (i.e. negative too) 3 Cycles Does a

831 views • 43 slides
Fractional Factorial Designs Each replicate of a 2 k design requires 2 k runs. E.g. 64 runs for k =

Fractional Factorial Designs Each replicate of a 2 k design requires 2 k runs. E.g. 64 runs for k =

ST 516 Experimental Statistics for Engineers II Fractional Factorial Designs Each replicate of a 2 k design requires 2 k runs. E.g. 64 runs for k = 6, or 1024 runs for k = 10. When this is infeasible, we use a fraction of the runs. As a result, we

580 views • 18 slides
Sampling Distribution of a Statistic Recall: a statistic is a summary calculated from a sample.

Sampling Distribution of a Statistic Recall: a statistic is a summary calculated from a sample.

ST 430/514 Introduction to Regression Analysis/Statistics for Management and the Social Sciences II Sampling Distribution of a Statistic Recall: a statistic is a summary calculated from a sample. Statistics vary from sample to sample. If samples

743 views • 21 slides
Repeatable Runs for Test Collection Documentation Ian Soboroff, NIST SIGIR 2019 OSIRRC Workshop,

Repeatable Runs for Test Collection Documentation Ian Soboroff, NIST SIGIR 2019 OSIRRC Workshop,

https://commons.wikimedia.org/wiki/File:Longshoreman,_Brooklyn,_ca._1872-1887._(5832938139).jpg Repeatable Runs for Test Collection Documentation Ian Soboroff, NIST SIGIR 2019 OSIRRC Workshop, Paris, France My questions for OSIRRC Can we

452 views • 6 slides
Evaluation of the GMTED2010 (250m) to derive topographic variables at 1km resolution (first

Evaluation of the GMTED2010 (250m) to derive topographic variables at 1km resolution (first

Evaluation of the GMTED2010 (250m) to derive topographic variables at 1km resolution (first results) Giuseppe Amatulli GMTED2010 7.5 arc-seconds (250m) Median Statistic Minimum Statistic Mean Statistic Maximum Statistic

231 views • 6 slides
A two-sample test for comparison of long memory parameters F. Lavancier 1 , A. Philippe 1 , D.

A two-sample test for comparison of long memory parameters F. Lavancier 1 , A. Philippe 1 , D.

Introduction Test Statistic Consistency bi-fBm bi-linear models In practice Simu A two-sample test for comparison of long memory parameters F. Lavancier 1 , A. Philippe 1 , D. Surgailis 2 1 Laboratoire Jean Leray, Universit de Nantes 2

649 views • 46 slides
A two-sample test for comparison of long memory parameters F. Lavancier 1 , A. Philippe 1 , D.

A two-sample test for comparison of long memory parameters F. Lavancier 1 , A. Philippe 1 , D.

Introduction Test Statistic In practice Simulations A two-sample test for comparison of long memory parameters F. Lavancier 1 , A. Philippe 1 , D. Surgailis 2 1 Laboratoire Jean Leray, Universit de Nantes 2 Institute of Mathematics and

576 views • 34 slides
2 An Aqueduct Runs Through It 3 An Aqueduct Runs Through It 4 An Aqueduct Runs Through It 5

2 An Aqueduct Runs Through It 3 An Aqueduct Runs Through It 4 An Aqueduct Runs Through It 5

An Aqueduct Runs Through It A Series on the Future of Californias Water Supply Southern California Planning Congress in cooperation with the California Center for Land and Water Stewardship, Cal Poly Pomona Barry Lehrman Director, Aqueduct

658 views • 10 slides
Weighted graphs Weighted graphs Weighted graphs Weighted graphs Graphs with numbers, called

Weighted graphs Weighted graphs Weighted graphs Weighted graphs Graphs with numbers, called

Weighted graphs Weighted graphs Weighted graphs Weighted graphs Graphs with numbers, called weights , attached to each edge K08 - Often restricted to

504 views • 12 slides
Topic 21 Goodness of Fit Contingency Tables 1 / 11 Introduction Two-way Table The Hypothesis

Topic 21 Goodness of Fit Contingency Tables 1 / 11 Introduction Two-way Table The Hypothesis

Introduction Two-way Table The Hypothesis The Test Statistic Degrees of Freedom Topic 21 Goodness of Fit Contingency Tables 1 / 11 Introduction Two-way Table The Hypothesis The Test Statistic Degrees of Freedom Outline Introduction

588 views • 11 slides
Observing Facts Andreas Zeller 1 Reasoning about Runs Experimentation n controlled runs

Observing Facts Andreas Zeller 1 Reasoning about Runs Experimentation n controlled runs

Observing Facts Andreas Zeller 1 Reasoning about Runs Experimentation n controlled runs Induction n runs Observation 1 run Deduction 0 runs 2 2 Reasoning about Runs Observation 1 run Deduction 0 runs 3 3 Principles of

559 views • 9 slides
Folding Carton Point of Purchase Display Purchasing Small runs Large runs Combo runs

Folding Carton Point of Purchase Display Purchasing Small runs Large runs Combo runs

Folding Carton Point of Purchase Display Purchasing Small runs Large runs Combo runs VMI In House Graphic Design In House Structural Design Stock Items WHO ARE WE? Packaging Technologies Inc. 310 Courtland Ave.

602 views • 5 slides
Engineering Best Practices Test, test, test, and test some more; test as you go Start from a

Engineering Best Practices Test, test, test, and test some more; test as you go Start from a

Engineering Best Practices Test, test, test, and test some more; test as you go Start from a known working state, take small steps Make things visible (gdb, printf, logic analyzer) Methodical, systematic approach. Form hypotheses and perform

755 views • 38 slides
runs and dat aset s analysis of t he dat aset s remaining quest ions & work runs

runs and dat aset s analysis of t he dat aset s remaining quest ions & work runs

Diurnal cycle of deep convection over land status of the intercomparison case runs and dat aset s analysis of t he dat aset s remaining quest ions & work runs and datasets use of more realistic surf ace heat f luxes t o

520 views • 25 slides
Unit 5: Inference for categorical variables Lecture 3: Chi-square tests Statistics 101 Thomas

Unit 5: Inference for categorical variables Lecture 3: Chi-square tests Statistics 101 Thomas

Unit 5: Inference for categorical variables Lecture 3: Chi-square tests Statistics 101 Thomas Leininger June 14, 2013 Chi-square test of GOF Chi-square test of GOF 1 Weldons dice The chi-square test statistic The chi-square distribution

1k views • 97 slides
Web testing Image by C Watts What is web testing? Testing web applications Applications of which

Web testing Image by C Watts What is web testing? Testing web applications Applications of which

Web testing Image by C Watts What is web testing? Testing web applications Applications of which the client runs in a web browser In this lecture on web testing What to test How to test it What to test Back end Front end HTTP (server)

356 views • 31 slides
Chapter 8 Inferences Based on a Single Sample: Tests of Hypothesis The Elements of a Test of

Chapter 8 Inferences Based on a Single Sample: Tests of Hypothesis The Elements of a Test of

Chapter 8 Inferences Based on a Single Sample: Tests of Hypothesis The Elements of a Test of Hypothesis 7 elements 1. The Null hypothesis 2. The alternate, or research hypothesis 3. The test statistic 4. The rejection region 5. The

478 views • 26 slides
Thanks to our Sponsors A brief history of Protg 1987 PROTG runs on LISP machines

Thanks to our Sponsors A brief history of Protg 1987 PROTG runs on LISP machines

Thanks to our Sponsors A brief history of Protg 1987 PROTG runs on LISP machines 1992 PROTG-II runs under NeXTStep 1995 Protg/Win runs under guess! 2000 Protg-2000 runs under Java 2005 Protg 3.0

815 views • 65 slides
Chi-square test on candidate events from CW signals coherent searches (Y. Itoh,

Chi-square test on candidate events from CW signals coherent searches (Y. Itoh,

Chi-square test on candidate events from CW signals coherent searches (Y. Itoh, M.A.Papa,B.Krishnan-AEI, X. Siemens UWM r A large value of the detection statistic indicates a F ( f k l , ) s r candidate signal at the

598 views • 12 slides
Regression models P R AC TIC IN G STATISTIC S IN TE R VIE W QU E STION S IN P YTH ON Conor De w

Regression models P R AC TIC IN G STATISTIC S IN TE R VIE W QU E STION S IN P YTH ON Conor De w

Regression models P R AC TIC IN G STATISTIC S IN TE R VIE W QU E STION S IN P YTH ON Conor De w e y Data Scientist , Sq u arespace Getting started 1 Wikimedia PRACTICING STATISTICS INTERVIEW QUESTIONS IN PYTHON Ass u mptions Linear

701 views • 51 slides
Introd u ction to statistical seismolog y C ASE STU D IE S IN STATISTIC AL TH IN K IN G J u

Introd u ction to statistical seismolog y C ASE STU D IE S IN STATISTIC AL TH IN K IN G J u

Introd u ction to statistical seismolog y C ASE STU D IE S IN STATISTIC AL TH IN K IN G J u stin Bois Lect u rer , Caltech California mo v es and shakes 1 Fa u lt data : USGS Q u aternar y Fa u lt and Fold Database of the United States CASE

970 views • 39 slides
Probabilit y densit y f u nctions STATISTIC AL TH IN K IN G IN P YTH ON ( PAR T 1 ) J u stin

Probabilit y densit y f u nctions STATISTIC AL TH IN K IN G IN P YTH ON ( PAR T 1 ) J u stin

Probabilit y densit y f u nctions STATISTIC AL TH IN K IN G IN P YTH ON ( PAR T 1 ) J u stin Bois Lect u rer at the California Instit u te of Technolog y Contin u o u s v ariables Q u antities that can take an y v al u e , not j u st discrete v

1.17k views • 45 slides
Conditional probabilities P R AC TIC IN G STATISTIC S IN TE R VIE W QU E STION S IN P YTH ON

Conditional probabilities P R AC TIC IN G STATISTIC S IN TE R VIE W QU E STION S IN P YTH ON

Conditional probabilities P R AC TIC IN G STATISTIC S IN TE R VIE W QU E STION S IN P YTH ON Conor De w e y Data Scientist , Sq u arespace Co u rse o v er v ie w 1. Probabilit y and sampling distrib u tions 2. E x plorator y data anal y sis 3.

521 views • 28 slides

More recommend