Statistical methods Tomaž Podobnik Oddelek za fiziko, FMF, UNI-LJ , , Odsek za eksperimentalno fiziko delcev, IJS
I Prologue: I. Prologue: 1. Literature, 2. Basic Desiderata, 3. Example. 3. Example. 4/15/2010 2
1. Literature: • no single textbook covers all topics appropriately, • relevant references will be given for each of the topics separately. 4/15/2010 3
2. Basic Desiderata: A Every system must be (self-) consistent. B Every scientific (empirical) theory must be operational (i.e., it must specify operations that ensure falsifiability ( , p y p y of its predictions). K Popper (1959) The Logic of Scientific Discovery § 24 pp 91-92 K. Popper (1959), The Logic of Scientific Discovery, § 24, pp. 91 92. J. Neyman (1937), Phil. Trans. R. Soc., A 236 , 333-380. G. Pólya (1954), Mathematics and Plausible Reasoning, Vol. 2 – Patterns of Plausible Inference Chap XIV § 4 p 64 Patterns of Plausible Inference, Chap. XIV, § 4, p. 64, and Chap. XV, § 4, p. 117. 4/15/2010 4
Definition 1 (Inconsistency). A system is inconsistent iff, within the Definition 1 (Inconsistency). A system is inconsistent iff, within the rules of the system, a conclusion can be reasoned out in more than one ways, and not all the ways lead to the same result. Definition 2 (Consistency). A system that is not inconsistent (i.e., a system without a single demonstrated inconsistency) is said a system without a single demonstrated inconsistency) is said to be consistent. 4/15/2010 5
3 Example 1 A decay of an atom (nucleus particle) at time t : 3. Example 1. A decay of an atom (nucleus, particle) at time t 1 : ⎧− ⎫ 1 t ( ) τ = ) ⎨ 1 ⎬ f I t | exp ; 1 τ τ τ τ 1 ⎩ ⎩ ⎭ ⎭ = = τ { } [ ] ( ) − + = τ = τ − τ τ ∈ 1 ( | I f t | exp t / ; t f I ( ) τ = P A | 1 t = ∫ b ( ( ) ) ( ( ) ) ( ( ) ) I ∫ τ τ = P A | | f f t | | dt ; ; A t , , t I I I I a a b b t t t t a b t t a 4/15/2010 6
τ ? Given t 1 , what is the value of the parameter The Maximum Likelihood Method (MLM): R A Fisher (1922) Phil Trans R Soc R. A. Fisher (1922), Phil. Trans. R. Soc., A 222 , 309-368. A 222 309 368 ( ) ( ) τ ≡ τ Likelihood function : . L t ; f t | 1 I 1 “The principle of Maximum Likelihood states that when confronted with The principle of Maximum Likelihood states that, when confronted with τ a choice of we choose that one (if any) which maximizes L .” A St art J K Ord (1994) Kendall’s Ad anced Theor of Statistics A. Stuart, J. K. Ord (1994), Kendall’s Advanced Theory of Statistics, Vol. 1 – Distribution Theory, § 8.22, p. 300. I. Kuš č er, A. Kodre (1994), Matematika v fiziki in tehniki, § 11.8, p. 309. R. Jamnik (1975), Verjetnostni ra č un in statistika, § 24, p. 529-530. In I. Vidav (ed.), Višja matematika II . 4/15/2010 7
General method: ∂ ( ) τ τ = ˆ : ln L t ; 0 τ = τ ) ˆ ∂ τ 1 τ ; 1 τ = ˆ ˆ t ( 1 = 1 t L τ What justifies MLM? 4/15/2010 8
τ ˆ (The fallacy of) The limiting property of : Set of measurements { t 1 ,…,t n } : i.i.d. n ( ) ( ) ( ) ∏ τ = τ = τ L t ,..., t ; f t ,..., t | f t | 1 n I 1 n I i = i 1 ∂ ∂ n 1 1 ( ) ( ) ∑ τ = τ τ = ⇒ τ = = ˆ ˆ t ,..., t : ln L t ,..., t ; 0 ˆ t t τ = τ ˆ ∂ τ 1 n 1 n n i n = i 1 τ ⎛ ⎞ ( ( ) ) τ τ τ τ τ τ ⎜ ⎜ ⎟ ⎟ ˆ lim lim : : f f | | ~ N N , ⎝ ⎠ → ∞ n n = ≈ ∞ = ≈ ∞ N 3 , n 1 C 4/15/2010 9
The conditional probability fallacy: ( ) ( ) τ τ = τ τ = Interested in calculate : f | t max ., : f t | max . 1 1 ( ( ) ) ( ( ) ) τ = τ ⇒ an implicit p assumption p f f | | t f f t | | : I I 1 1 I I 1 1 ∞ ( ( ) ) ∫ ∫ τ d τ = ∞ f f I t 1 | | d 0 Example (L. Lyons): A : ”The first person I’ll meet is female.” B : ”The first person I’ll meet is pregnant.” B ”Th fi t I’ll t i t ” ( ) ( ) ≠ P A | B P B | A 4/15/2010 10
The fallacy of point estimates: τ The parameter may take on every value in a continuum p y y τ ˆ + ï a measure of a single point in the continuum is 0. For verifiable predictions we must turn to interval estimations For verifiable predictions we must turn to interval estimations. “Ignorance is preferable to error and he is less remote from the truth who belives nothing than he who believes what is wrong.” g g (T. Jefferson (1781). Notes on Virginia .) 4/15/2010 11
Contents: I. Prologue, II. Mathematical Preliminaries, III. Frequency Interpretation of Probability Distributions, IV. Confidence Intervals, V. Testing of Hypotheses, VI. Inverse Probability Distributions, VII. Interpretation of Inverse Probability Distributions, VIII.Time Series and Dynamical Models. 4/15/2010 12
II. Mathematical Preliminaries: 1. Motivation, 2. Probability spaces, 3. Conditional probabilities, 4. Random variables, 5. Probability distributions, 6. Transformations of probability distributions, 7. Conditional distributions, 8. Parametric families of (direct) probability distributions, 9. The Central limit Theorem, 10.Invariant parametric families. 4/15/2010 13
1. Motivation : • (self-) consistency, “ Jede axiomatische (abstrakte) Theorie läßt bekanntlicht • unbegrentzt viele konkrete Interpretationen zu.” (A. N. Kolmogorov (1933). Grundbegriffe der Wahrscheinlichkeitsrechnung, Chap. I, p. 1.) g, p , p ) A. Stuart, J. K. Ord (1994), Kendall’s Advanced Theory of Statistics, Vol. 1 – Distribution Theory . M M Rao (1993) Conditional Measures and Applications M.M.Rao (1993), Conditional Measures and Applications. 4/15/2010 14
2. Probability spaces: Ω = abstract, non - empty) universal set ( Ω Ω ⊆ Ω K A , B , C { } Σ = K A , B , C , A A C Definition 3 (s -algebra ). S is called s -algebra ( s -field) on W iff: Ω ∈ Σ i) , B Ω ∈ Σ ii) \ A ; A , ∑ ∑ ∞ ∞ ∈ Σ ∈ Σ iii) A ; A . i i = i 1 Definition 4 (Measurable space). An ordered pair (W,S) , consisting of a universal set W and a s - algebra S on W is called a measurable space. 4/15/2010 15
Example 2 (Measurable space). (W,S) , where S={W,«} . Example 3 (Measurable space). ( n , n ),where n is the Borel s -algebra (the smallest s -algebra containing all n-dimensional open rectangles). Definition 5 (Probability measure). A real-valued function P on a s -algebra S is called Probability measure iff: g y ( ) ≥ ∈ Σ i) P A 0 ; A , ( ) Ω = ii) P 1 , ( ) ( ) ∑ ∞ ∞ = = ∅ iii) U I P A P A ; A A . = ≠ i 1 i = i i j i i 1 Definition 6 (Probability space). An ordered triple (W,S, P ) , consis- ting of a universal set W , of a s -algebra S on W , and of a probabi- lity (measure) P on S is called probability space. lity (measure) P on S is called probability space. 4/15/2010 16
3. Conditional probability: Definition 7 (Conditional probability). Consider a probability space p y p ( p y) (W,S, P ) and sets A,B œS, P ( B )>0. Then, the conditional probability P ( A|B ) is defined as ( ( ) ) I I P A B ( ( ) ) ≡ P P A A | | B B . ( ) P B Definition 8 (Independent sets). Consider a probability space t ) C id b bilit D fi iti 8 (I d d t (W,S, P ) and sets A,B œS. The sets are ( P -) independent iff ( ( ) ) = I P A B P ( ( A ) ) P ( ( B ) ) . Definition 9 (Mutually exclusive and exhaustive sets). Consider a space (W S P ) and a set { B space (W,S, P ) and a set { B 1 ,..., B n }, B i œS and P ( B i )>0 . When B } B œS and P ( B )>0 When = ∅ = Ω and n I U B B B , ≠ = i j i i 1 i the sets B i are called mutually exclusive and (W - ) exhaustive . ( ) y i 4/15/2010 17
Proposition 1 (Law of Total Probability, LTP). Let (W,S, P ) be a probability space and { B j } mutually exclusive and W -exhaustive sets Then P ( A ) A œS decomposes as sets. Then, P ( A ), A œS, decomposes as ( ) ( ) . ( ) ∑ = n = P A P B P A | B j j j 1 ( ( ( ( ) ) ) ) ( ( ( ( ) ) ) ) ( ) ( ) ( ( ) ) = Ω = = n n I I U U I P A P A P A B P A B Proof. = = j 1 j j 1 j ( ) ( ) ( ) . ∑ ∑ n n = = I P A B P B P A | B = j = j j j j 1 j j 1 Theorem 1 (Bayes). Let (W,S, P ) be a probability space, { B j } mutually exclusive and W exhaustive sets and P ( A )>0 A œS Then exclusive and W -exhaustive sets, and P ( A )>0, A œS. Then, ( ) ( ) P B P A | B ( ) = i i P B | A . ( ) ( ( ) ( ) ) ∑ ∑ = i n P P B B P P A A | | B B j j j j j 1 ( ) ( ) ( ) ( ) ( ) = = Proof. follows from Def. 7, P A I B P B P A | B P A P B | A i i i i whereas P ( A ) decomposes according to LTP. p g ( ) 4/15/2010 18
Recommend
More recommend
