Statistical methods Tomaž Podobnik Oddelek za fiziko, FMF, UNI-LJ , , Odsek za eksperimentalno fiziko delcev, IJS
Contents: I. Prologue, II. Mathematical Preliminaries, III. Frequency Interpretation of Probability Distributions, IV. Confidence Intervals, V. Testing of Hypotheses, VI. Inverse Probability Distributions, VII. Interpretation of Inverse Probability Distributions, VIII.Time Series and Dynamical Models. 4/15/2010 2
VI. Inverse Probability Distributions: 1. Direct probability distributions (a refresher), p y ( ), 2. Inverse probability distributions, 3. (Non-informative) Prior probability distributions, ( ) p y 4. Factorization Theorem and consistency factors, 5. Objectivity, j y 6. Consistency factors for location-scale families, 7. Consistency factors in the absence of symmetry. y y y 4/15/2010 3
1. Direct probability distributions: Definition 21 (Parametric family). The term parametric family stands { } = θ ∈ Pr , θ : for a collection of probability distributions that I V Θ I differ only in the value q of a parameter Q. y p F ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) = = = , , ; F x F F x f f x f f x p x p x θ θ θ , , , X I X I X I ( ) ( ) ( ( ) ) ( ) ( ) ( ( ) ) ( ) ( ) ( ( ) ) . = θ θ = θ θ = θ θ | | , | | , | | F F x x F F x x f f x x f f x x p p x x p p x x θ θ θ , , , I I I I I I ( ) ( ) ( ) ( ) ∫ = = θ = θ ∈ Example 1. Pr Pr Pr | | , . B B B f x dx B θ , X I I I B 4/15/2010 4
Definitions extend without change to random vectors: ( ) ( ) ( ) ( ) ( ) ( ) ; = = = x x | θ , x x | θ , x x | θ F F f f p p X X X I I I ( ) ( ) ( ) ( ) ∫ = = = ∈ . Pr Pr Pr | θ x | θ x ; n n B B B f d B X , θ I I I B Conditional probability distributions: ( ( ) ) ( ( ) ) ( ( ) ) ( ( ) ) ∫ ∫ = = ≡ > X ( ( , , ) ) , , , , , , | | θ , , | | θ , , | | θ 0 Y Z f f y y z f f y y z f f z f f y y z dy y X X I I I I I I ( ) , | θ ( ) f y z ⇒ ≡ | , θ ) . I f y z ( I | θ f z I Example 2 (Reparametrization). Let a probability distribution for a ( ) ( ) ( ( ) , ) , = continuous random vector X belong to a family I, g y x x | θ | f f f f X X I I ( ) ( ) ( ) ( ) ≡ ≡ ∂ ∂ ≠ and let and with y s x λ s θ s x , s θ 0 . Then, x θ ( ) ( ) = − − ∂ − 1 1 1 y | λ s ( y ) | s ( λ ) s ( y ) . f f ' y I I 4/15/2010 5
2. Inverse probability distributions: Definition 1 (Inverse probability distribution). Given a probability Definition 1 (Inverse probability distribution) Given a probability ( ) ( ) Ω → ∈ space (W,S, P ) and a function Θ , X : , x , x , Θ is m n , ( ) , Σ − the inverse probability distribution measurable , and X ~ x | θ f I Θ Θ , i d fi is defined as the image measure of P by the random variable d th i f P b th d i bl ( ) • ≡ − 1 Pr | x Θ , o such that P [ [ ] ] I ( ( ) ) − = ∈ 1 Pr | | x Θ ( ( ) ) , , . m B P B B I I Distribution and density functions: ( ) ( ) ( ) ( ) . ↔ ↔ x | θ θ | x , x | θ θ | x F F f f I I I I Also: probability distributions that are neither purely direct nor purely p y p y p y inverse. Distribution and density functions of such a distribution: ( ) ( ) . θ , x | x and θ , x | x F f 1 2 1 2 I I 4/15/2010 6
From a mathematical perspective, all probability distributions, be they purely direct, purely inverse, or mixtures of the two, are equivalent. p y , p y , , q Conditional inverse probability distributions: Conditional inverse probability distributions: ( ) ( ) ( ) ∫ = Θ Θ ∃ ≡ > θ θ θ θ θ θ Θ ( , ) , , | x , | x , | x 0 f f f d 1 2 1 2 2 1 2 1 I I I ( ( ) ) θ θ 1 θ θ , , | | x x ( ( ) ) f f ⇒ θ θ θ θ ≡ | | , x 1 2 2 ) . I I f f ( 1 2 θ I | x f 2 I ( ) , Example 2 (Reparametrization). Suppose there is and let θ | x f ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) I ≡ ≡ ≡ ≡ ∂ ∂ ∂ ∂ ≠ ≠ y y s s x x and and λ λ s s θ θ with with s s x x , s s θ θ 0 0 . Then Then, x θ θ ( ) ( ) = − − ∂ − 1 1 1 λ | y s ( λ ) | s ( y ) s ( λ ) . f f ' λ I I 4/15/2010 7
3. (Non-informative) Prior probability distributions: ( ) ( ) ( ) ( ) ( ) = = Theorem 1 (Bayes). θ , x θ | x x x | θ θ f f f f f I I I I I ( ) ( ) ( ( ) ) θ θ x x | | θ θ ( ) f f f f ( ) ( ) ( ) ∫ ∫ ⇒ = = θ | x , x θ x | θ θ . m I I f f f f d ( ) I x I I I m f I T. Bayes (1763). Phil. Trans. R. Soc., 53 , 370 418. T Bayes (1763) Phil Trans R Soc 53 370-418 P. S. Laplace (1774). Mém. Acad. R. Sci., 6 , 621-656. ( ) θ : (non - informativ e) prior probabilit y distributi on. f I ( ) Uniform θ : (Bayes, Laplace). f I 4/15/2010 8
Mathematical difficulties: ∞ ∞ ) ) ( ) ( ) ( ) ( ) ( ) ( ) ( ( ) ) ∫ ∫ ∫ ∫ τ τ = ⇒ ⇒ τ τ τ τ = ∞ ∞ τ τ τ τ τ τ = ∞ ∞ const const. , | | , a a f f f f d d f f f f t t d d 1 1 I I I I I I I I 0 0 ) ( ) μ μ = τ ⇒ μ = ≠ ln const. b f e ′ I σ = 1 ) τ ; 1 1 = ( 1 t L τ 4/15/2010 9
Conceptual (interpretational) difficulties: τ ↔ a) a) We We do do not not know know anything anything about about we we know know that that τ is uniformly distribute d, τ ↔ τ b) is unknown but fixed is distribute d. “ ...inverse probability is a mistake (perhaps the only mistake p y (p p y to which the mathematical world has so deeply committed ï itself),...” (R. A. Fisher (1922). Phil. Trans. R. Soc., A 222 , 309-368) (R. A. Fisher (1922). Phil. Trans. R. Soc., A 222 , 309 368) 4/15/2010 10
4. Factorization Theorem and consistency factors: ( ( ) ) ( ( ) ; ) ∃ ∃ Theorem 2 (Bayes) Theorem 2 (Bayes). θ θ , x x | | x x , θ θ , x x | | x x ; f f f f 1 2 2 1 I I ( ) ( ) ∫ = > θ | x θ , x | x x 0 ; n f f d 1 , 2 2 1 2 , 1 I I n ( ( ) ) = ∫ ( ( ) ) ∫ > ; x | x θ , x | x m θ 0 f f d 1 , 2 2 , 1 2 1 I I m ( ) ( ) ( ) = X , X i.i.d., x , x | θ x | θ x | θ ; f f f 1 2 1 2 1 2 I I I ( ( ) ) ( ( ) ) ( ( ) ) ( ( ) ) θ | x x | θ θ | x x | θ ( ) f f f f ⇒ = = θ | x , x 1 2 2 1 . I I I I f ( ) ( ) 1 2 I x | x x | x f f 2 1 1 2 I I Under Under analogous analogous conditions conditions : : ( ) ( ) ( ) ( ) θ | θ , x x | θ , θ θ | θ , x x | θ , θ ( ) f f f f = = 1 2 1 2 1 2 1 2 2 1 1 2 . θ | θ , x , x I I I I f ( ( ) ) ( ( ) ) 1 2 1 2 I x x | | θ θ , , x x x x | | θ θ , , x x f f f f 2 2 2 2 1 1 1 1 2 2 2 2 I I I I ( ( ) ) ( ( ) ( ) ( ) ) ( ( ) ( ) ( ) . ) = = Proof. θ , x | | x θ | | x x | | θ x | | x θ | | x , x f f f f f f f f f f 1 1 , 2 2 2 2 , 1 1 2 2 , 1 1 1 1 , 2 2 1 1 , 2 2 2 2 , 1 1 1 1 2 2 I I I I I I I I I I 4/15/2010 11
Recommend
More recommend
