Statistical significance in CP violation Mattias Blennow emb@kth.se - PowerPoint PPT Presentation

Statistical significance in CP violation Mattias Blennow emb@kth.se KTH Theoretical Physics June 22, 2015, Invisibles 15, Madrid, Spain Mattias Blennow KTH Theoretical Physics Statistical significance in CP violation

Parameter estimation sensitivity Define the test statistic “∆ χ 2 ” � L ( θ | d ) � ∆ χ 2 ( θ ) = − 2 log sup θ ′ L ( θ ′ | d ) Assume it is χ 2 distributed with n degrees of freedom Use the data set without statistical fluctuations (Asimov data) Quote result Mattias Blennow KTH Theoretical Physics Statistical significance in CP violation

The interpretation Probability to observe a non-zero θ 13 at 99.73% CL in T2K 2 π ∆ χ 2 is asymptotically χ 2 < 20% 20%-50% 3 π /2 (Wilks’ theorem) 50%-90% 90%-99% > 99% true δ CP The Asimov data (expected π data without fluctuations) is representative π/2 Several requirements, not 0 always fulfilled -3 -2 -2 3 . 10 7 . 10 10 2 2 θ 13 true sin Schwetz, Phys.Lett. B648 (2007) 54 Mattias Blennow KTH Theoretical Physics Statistical significance in CP violation

For the mass ordering NH MC 0.1 ) 2 χ IH MC ∆ Mass ordering is not nested NH Norm. Approx. PDF( 0.08 IH Norm. Approx. 0.06 Wilks’ theorem not applicable 0.04 Test statistic 0.02 χ 2 IO − χ 2 0 T = -40 -20 0 20 40 NO 2 ∆ χ Qian, et al. , Phys.Rev. D86 (2012) 113011 T is approximately Gaussian for many situations � T ≃ N ( T 0 , 2 T 0 ) T 0 = value for Asimov data Mattias Blennow KTH Theoretical Physics Statistical significance in CP violation

What is sensitivity? Interpretation: It is representative for how well the Sensitivity (median) experiment will do What is the 50 % probability of not reaching it expected rejection 50 % probability of doing better of a false ordering? Not 50 % probability of “being wrong” (Given a parameter set) Not the only relevant quantity, distribution matters (do Brazilian bands!) Mattias Blennow KTH Theoretical Physics Statistical significance in CP violation

Mass ordering results NOvA MC � Β� 0.5 � 6 6 Gaussian � 1 � sided � 6 Gaussian � 2 � sided � Sensitivity � Σ � Α�Β 5 5 4 median (2 sided) median (1 sided) 2 4 4 sensitivity ( σ ) 0 � 150 � 100 � 50 0 50 100 150 3 3 crossing (2 sided) ∆ � ° � crossing (1 sided) 2 2 LBNE � 10kt MC � Β� 0.5 � Gaussian � 1 � sided � 6 Gaussian � 2 � sided � Sensitivity � Σ � 1 1 Α�Β 4 0 0 0 0 10 10 20 20 30 30 2 T 0 MB, Coloma, Huber, Schwetz, JHEP 03(2014)028 0 � 150 � 100 � 50 0 50 100 150 ∆ � ° � Mattias Blennow KTH Theoretical Physics Statistical significance in CP violation

Other measurements CP violation θ 23 octant Nested hypothesis Degeneracies closer Does not mean that Wilk’s Wilk’s theorem still violated theorem holds, cyclic A priori, a dedicated study is parameters needed Rest of this talk Mattias Blennow KTH Theoretical Physics Statistical significance in CP violation

Setup for CP violation Test statistic 0.1 δ =0 ,π χ 2 − min 0.09 global χ 2 S = min 0.08 0.07 0.06 P e µ 0.05 Why not necessarily gaussian? 0.04 0.03 Cyclic parameter 0.02 0.01 Several points in null hypothesis 0 0 0.02 0.04 0.06 0.08 0.1 ( δ = 0 , π ) P e µ Blennow, Smirnov, Adv.High Energy Phys. Degeneracies 2013 (2013) 972485 295 km, 0.65 GeV Distribution should always be checked or argued for Mattias Blennow KTH Theoretical Physics Statistical significance in CP violation

Critical values Expectation from null hypothesis 1 1 Σ Red line shows the χ 2 distribution 0.1 2 Σ 1 � CDF NOvA to lower cutoff LBNE T2HK values 0.01 NO Ν A ESS More sensitive 3 Σ experiments to higher T2HK � 20 Χ 2 0.001 0 2 4 6 8 10 cutoff values S MB, Coloma, Fernandez-Martinez, JHEP 1503 (2015) 005 Mattias Blennow KTH Theoretical Physics Statistical significance in CP violation

Median deviations Need to consider the expected outcome 70 60 Median ESS Agrees quite well for 50 Asimov most experiments 40 S T2HK 30 Lower than Asimov data for NOvA 20 LBNE 10 Depending on δ for NO Ν A 0 other experiments 0 45 90 135 180 225 270 315 360 ∆ MB, Coloma, Fernandez-Martinez, JHEP 1503 (2015) 005 Mattias Blennow KTH Theoretical Physics Statistical significance in CP violation

Sensitivity results Combining the distributions with the cutoffs 3.0 3.0 NO Ν A 2.5 2.5 Median 2.0 Asimov 2.0 T2HK 1.5 1.5 Σ Σ Median 1.0 1.0 Asimov 0.5 0.5 0.0 0.0 0 45 90 135 180 225 270 315 360 0 45 90 135 180 225 270 315 360 ∆ ∆ 3.0 3.0 2.5 2.5 2.0 2.0 ESS 1.5 LBNE 1.5 Σ Σ Median 1.0 1.0 Asimov Median 0.5 Asimov 0.5 0.0 0.0 0 45 90 135 180 225 270 315 360 0 45 90 135 180 225 270 315 360 ∆ ∆ MB, Coloma, Fernandez-Martinez, JHEP 1503 (2015) 005 Mattias Blennow KTH Theoretical Physics Statistical significance in CP violation

Handling of nuisance parameters Systematics and previous measurements: Addition to the χ 2 function 0 ( θ, ξ ) + ( ξ − ¯ ξ ) 2 χ 2 ( θ, ξ ) = χ 2 σ 2 ξ ξ is the fit value of the nuisance parameter ¯ ξ is the experimental measurement or theoretical prediction A priori: Should calibrate χ 2 for all true values of ξ true In reality: Little dependence on the true value, calibrate for ξ true = ¯ ξ for existing experiments Mattias Blennow KTH Theoretical Physics Statistical significance in CP violation

Current hints 2 levels NuFit 2.0 (2014) observed data ∆χ 8 2 θ 23 = 0.4 2 θ 23 = 0.5 2 θ 23 = 0.6 sin sin sin 7 6 99% 99% 5 99% 2 ∆χ 4 95% 3 95% 95% 90% 90% 2 90% 1 68% 68% 68% 0 0 90 180 270 0 90 180 270 0 90 180 270 360 δ CP δ CP δ CP Gonzalez-Garcia, Maltoni, Schwetz, JHEP 1411 (2014) 052 www.nu-fit.org, 2014 Mattias Blennow KTH Theoretical Physics Statistical significance in CP violation

Heuristic interpretation Parameter space is curved N Ν Do not expect χ 2 Can we understand the deviations? N Ν 1 Large errors on rates: Difference 1 Σ between best fit and null hypothesis 0.1 2 Σ small 1 � CDF 0.01 s � 1 s � 0.3 Medium errors: Curvature plays a role 3 Σ s � 0 0.001 Small errors: Possible outcomes 0 2 4 6 8 10 S essentially linearly related to δ Mattias Blennow KTH Theoretical Physics Statistical significance in CP violation

Summary and conclusions Wilks’ theorem is not a priori applicable to the neutrino CP violation, the test statistic is not χ 2 distributed More precise experiments → χ 2 Critical values will depend on the experiments Generally: Lower critical values for low precision experiments Also expect lower χ 2 than Asimov for those The usual Asimov + χ 2 approximation is a relatively good estimator Mattias Blennow KTH Theoretical Physics Statistical significance in CP violation