Statistical methods Toma Podobnik Oddelek za fiziko, FMF, UNI-LJ , - - PowerPoint PPT Presentation

SLIDE 1

Statistical methods

Tomaž Podobnik Oddelek za fiziko, FMF, UNI-LJ , , Odsek za eksperimentalno fiziko delcev, IJS

SLIDE 2

Contents:

I. Prologue, II. Mathematical Preliminaries,

• III. Frequency Interpretation of Probability Distributions,
• IV. Confidence Intervals,

V. Testing of Hypotheses,

• VI. Inverse Probability Distributions,
• VII. Interpretation of Inverse Probability Distributions,

VIII.Time Series and Dynamical Models.

4/15/2010 2

SLIDE 3

• VI. Inverse Probability Distributions:
• 1. Direct probability distributions (a refresher),

p y ( ),

• 2. Inverse probability distributions,
• 3. (Non-informative) Prior probability distributions,

( ) p y

• 4. Factorization Theorem and consistency factors,
• 5. Objectivity,

j y

• 6. Consistency factors for location-scale families,
• 7. Consistency factors in the absence of symmetry.

y y y

4/15/2010 3

SLIDE 4

• 1. Direct probability distributions:

Definition 21 (Parametric family). The term parametric family stands for a collection of probability distributions that differ only in the value q of a parameter Q.

{ }

Θ

∈ = V I

I

θ

θ :

Pr , y p

( ) ( ) ( ) ( ) ( ) ( )

f f F F ( )

( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )

| | | ; , ,

, , ,

θ θ θ

θ θ θ

x p x p x f x f x F x F x p x p x f x f x F x F

I X I X I X

= = =

( ) ( ) ( ) ( ) ( ) ( ).

| , | , |

, , ,

θ θ θ

θ θ θ

x p x p x f x f x F x F

I I I I I I

= = = Example 1.

( ) ( ) ( ) ( )

.  ∈ = = =

∫

B dx x f B B B

B I I I X

, | | Pr Pr Pr

,

θ θ

θ

4/15/2010 4

SLIDE 5

Definitions extend without change to random vectors:

( ) ( ) ( ) ( ) ( ) ( );

| , | , | θ x x θ x x θ x x

X X X I I I

p p f f F F = = =

( ) ( ) ( ) ( )

.

n n B I I I

B d f B B B  ∈ = = =

∫

; | | Pr Pr Pr

,

x θ x θ

θ X

Conditional probability distributions:

( ) ( ) ( ) ( )

| , | , | , , , ) , ( > ≡ = =

∫

dy z y f z f z y f z y f Z Y

I I I

θ θ θ X

X

( ) ( ) ( ) .

θ θ θ | | , , | z f z y f z y f

I I I

≡ ⇒

( ) ( ) ( ) ( )

| , | , | , , , ) , (

∫

y y f f y f y f

I I I  X

Example 2 (Reparametrization). Let a probability distribution for a continuous random vector X belong to a family I,

( ) ( ),

|θ x x

X I

f f = g y and let Then,

( )

( )

. ) ( ) ( | ) ( |

1 1 1 '

y s λ s y s λ y

y − − −

∂ =

I I

f f

( ) ( ),

|

X I

f f

( ) ( ) ( ) ( )

. , ≠ ∂ ∂ ≡ ≡ θ s x s θ s λ x s y

θ x

with and

4/15/2010 5

SLIDE 6

• 2. Inverse probability distributions:

Definition 1 (Inverse probability distribution) Given a probability Definition 1 (Inverse probability distribution). Given a probability space (W,S,P) and a function the inverse probability distribution i d fi d th i f P b th d i bl

( )

( )

is ,Θ x x X Θ

n m

 ,  ∈ → Ω , , : ,

( ),

| and , measurable θ x X

I

f ~ − Σ

Θ

is defined as the image measure of P by the random variable such that

( )

,

1

| Pr

−

≡

• Θ

x

• P

I

, Θ

( )

[ ]

.

m I

B B P B  ∈ =

−

, ) ( | Pr

1

Θ x

( )

[ ]

I

, ) ( | Distribution and density functions: Also: probability distributions that are neither purely direct nor purely

( ) ( ) ( ) ( ).

x θ θ x x θ θ x | | , | |

I I I I

f f F F ↔ ↔ p y p y p y

• inverse. Distribution and density functions of such a distribution:

( ) ( ).

and

2 1 2 1

| , | , x x θ x x θ

I I

f F

4/15/2010 6

SLIDE 7

From a mathematical perspective, all probability distributions, be they purely direct, purely inverse, or mixtures of the two, are equivalent. p y , p y , , q Conditional inverse probability distributions: Conditional inverse probability distributions:

( ) ( )

x | , |

2 1 θ

θ θ θ

I

f f

( ) ( ) ( )

| , | , | , , ) , (

1 2 1 2 2 1 2 1

> ≡ ∃ Θ Θ =

∫

θ θ θ θ θ θ d f f f

I I I 

x x x Θ

( ) ( ) ( ) .

x x x | | , , |

2 2 1 2 1

θ θ θ θ θ

I I I

f f f ≡ ⇒ Example 2 (Reparametrization). Suppose there is and let Then

( ),

| x θ

I

f

( ) ( ) ( ) ( )

≠ ∂ ∂ ≡ ≡ θ s x s θ s λ x s y

θ

with and Then,

( )

( )

. ) ( ) ( | ) ( |

1 1 1 '

λ s y s λ s y λ

λ − − −

∂ =

I I

f f

( ) ( ) ( ) ( )

. , ≠ ∂ ∂ ≡ ≡ θ s x s θ s λ x s y

θ x

with and

4/15/2010 7

SLIDE 8

• 3. (Non-informative) Prior probability distributions:

Theorem 1 (Bayes).

( ) ( ) ( ) ( ) ( ) ( ) ( )

| | | ,

∫

= =

I I I I I

f f f f f f f θ x θ θ θ x x x θ x θ

T Bayes (1763) Phil Trans R Soc 53 370-418

( ) ( ) ( ) ( ) ( ) ( ) ( )

. | , | |

∫

= = ⇒

m

m I I I I I I I

d f f f f f f f



θ θ x θ x x θ x θ x θ

• T. Bayes (1763). Phil. Trans. R. Soc., 53, 370 418.
• P. S. Laplace (1774). Mém. Acad. R. Sci., 6, 621-656.

( )

• n.

distributi y probabilit prior e) informativ

• (non

: θ

I

f

( )

Laplace). (Bayes, : Uniform θ

I

f

4/15/2010 8

SLIDE 9

Mathematical difficulties:

) ( ) ( ) ( ) ( )

| const

1

∞ = ∞ = ⇒ =

∞ ∞

∫ ∫

τ τ τ τ τ τ d t f f d f f a

I I I I

) ( ) ( ) ( ) ( ) ) ( )

const. ln , | , const.

1

1

≠ = ⇒ = ∞ ∞ ⇒

=

′

∫ ∫

μ

μ τ μ τ τ τ τ τ τ

σ

e f b d t f f d f f a

I I I I I

)

τ ; 1

1 =

( 1

t L

τ

4/15/2010 9

SLIDE 10

Conceptual (interpretational) difficulties:

that know we about anything know not do We a) d, distribute uniformly is that know we about anything know not do We a) τ τ ↔ d. distribute is fixed but unknown is b) τ τ ↔

“ ...inverse probability is a mistake (perhaps the only mistake p y (p p y to which the mathematical world has so deeply committed itself),...” (R. A. Fisher (1922). Phil. Trans. R. Soc., A 222, 309-368) ï (R. A. Fisher (1922). Phil. Trans. R. Soc., A 222, 309 368)

4/15/2010 10

SLIDE 11

• 4. Factorization Theorem and consistency factors:

Theorem 2 (Bayes)

( ) ( );

| | x x θ x x θ f f ∃

Theorem 2 (Bayes).

( ) ( )

( )

( )

( )

( )

; ; ,

1 , 2 1 2 2 , 1 1 2 2 1

| , | | , | , x x x θ x θ x x θ x x θ

n I I I I

d f f f f

n

> = ∃

∫ ∫



( )

( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )

; i.i.d., ;

2 1 2 1 2 1 1 2 1 , 2 2 , 1

| | | , , | , | θ x θ x θ x x X X θ x x θ x x

I I I m I I

f f f d f f

m

= > = ∫



( ) ( ) ( ) ( ) ( ) ( ) ( )

: conditions analogous Under .

2 1 1 2 1 2 2 1 2 1

| | | | | | , | x x θ x x θ x x θ x x θ x x θ

I I I I I I I

f f f f f f f = = ⇒

( ) ( ) ( ) ( ) ( ) ( ) ( )

. : conditions analogous Under

2 2 1 2 1 1 2 2 1 1 2 2 2 1 2 1 2 1 2 1 2 1

, | , | , | , | , | , | , , | x θ x θ θ x x θ θ x θ x θ θ x x θ θ x x θ θ

I I I I I I I

f f f f f f f = =

฀

( ) ( )

( ) ( ) ( ) ( ) (

).

2 1 1 2 2 1 2 1 1 2 1 2 2 1 2 2 1 1 2 2

, | | | | | , , | , | x x θ x x θ x x θ x x θ x θ x x θ x

I I I I I I I

f f f f f f f = = Proof.

4/15/2010 11

฀

( ) ( ) ( ) ( ) (

)

2 1 1 , 2 2 , 1 2 , 1 1 , 2 1 , 2 2 , 1

| | | | |

I I I I I

f f f f f

SLIDE 12

Theorem 3 (Factorization).

( ) ( ) ( ) ( ) ( ) ( ) ( )

= =

1 2 2 1 2 1

| | | | |

I I I I I

f f f f f θ x x θ θ x x θ x x θ

( ) ( ) ( )

( ) ( ) ( )

∫

= = ⇒

2 , 1 2 , 1 2 , 1 2 , 1 2 1 1 2 2 1

| ) ( ) ( ; ) ( | ) ( | | | , |

m I I I I I I I I I

d f f f f f f . θ θ x θ x θ x θ x θ x x x x x x θ ζ η ζ

( ) ( )

( ) ( ) ( ) ( ) ( )

∫

, , 2 , 1 ,

| | | | ) (

I

f f f f : Similarly θ θ x x θ θ θ θ x x θ θ x η

( ) ( ) ( ) ( ) ( ) ( ) ( )

( ) ( ) ( )

∫

= =

1 2

2 1 2 , 1 1 , 2 2 1 2 1 1 2 2 1 1 2 2 2 1 2 1 2 1 2 1 2 1

| ) ( ) ( , | ) ( | , | , | , | , | , | , | , , |

m I I I I I I I I I

d f f f f f f f f f f θ θ θ θ θ θ x θ θ θ x θ x θ θ x x θ θ x θ x θ θ x x θ θ x x θ θ

θ

ζ ζ

( ) ( ) ( )

∫

= = ⇒

1 1 2 2 2 2

1 2 1 2 , 1 1 , 2 , 1 , 2 , 1 , , , 2 , 1 2 1

, | ) ( ) ( ; ) ( , |

m

m I I I I I

d f f



. θ θ θ x θ x x x θ θ

θ θ θ θ

ζ η η

( )

( )

| ) ( | ) ( ; ) ( ) , (

1 2 2 , 1 1

x x x x x θ x θ x x x θ

I I

f h f h κ κ ≡ ≡ = Proof ฀

( )

( )

. ) ( ) ( ) , ( ) ( ) , ( ) ( ) ( ) , ( | ) , ( , | ) , ( ; ) , ( ) , (

2 1 1 2 1 2 1 2 1 2 , 1 2 , 1 2 1 2

θ x θ x θ x x x x x x x θ x x θ x x x θ

I I I I

h f h f h ζ κ κ η κ κ = = ⇒ = ⇒ ≡ ≡ = Proof.

4/15/2010 12

฀ ) ( ) ( ) ( ) ( ) , (

2 1 2 2 1

x x x

I I I I

ζ η η η

SLIDE 13

Theorem 1 (Bayes) vs. Theorem 3 (Factorization):

( ) ( ) ( ) ( )

| ) ( | ) ( θ x θ θ x θ

I I I I

f f f ζ

( ) ( ) ( ) ( )

; ) ( ) ( ; ) ( | ) ( | ) ( | ) ( | θ θ x θ x θ x θ x θ x θ x θ

I I I I I I I I I I

f f f f f f f ζ η ζ ↔ = ↔ = ). integrable be not (need pdf a not ) (θ

I

ζ : Difference

( ) ( ) ( ) ( )

. : multiplier a to up

• nly

unique θ x θ θ x θ x x θ

I I I I I

f f

∫ ∫

= | ) ( | ) ( ) ( ) ( ) ( ζ ζ χ χ ζ

( ) ( )

. θ θ x θ θ θ x θ x

m I I m I I

d f d f

m m

∫ ∫

 

| ) ( | ) ( ) ( ζ ζ χ : ization reparametr under

• f

tion Transforma ) (θ

I

ζ

. , , ) ( ) ( ) ( )] ( [ ) (

'

≠ ∂ ≡ ∂ = λ s θ s λ λ s λ s λ

λ λ

• 1
• 1
• 1

I I

ζ ζ

4/15/2010 13

, , ) ( ) ( ) ( )] ( [ ) (

λ λ I I

ζ ζ

SLIDE 14

5 Obj ti it

• 5. Objectivity:

Definition 2 (Objectivity). A probabilistic parametric inference is called objective if a particular likelihood function always leads called objective if a particular likelihood function always leads to the same posterior density function. Motivation: at the beginning of the inference only the parametric family is known, and inferences based on identical information should be the same. Invariance:

( ) ( )

| | ; , ) , ( , ) , ( λ λ θ λ x y F F G a a a l l ∈ ≡ ≡

( ) ( ) ( )

( )

. ) , ( ) , ( | ) , ( | ; | |

'

y λ y λ y λ y λ y

y

a a a f f F F

I I I I 1

• 1
• 1
• l

l l ∂ = =

î Relative invariance of zI(q):

. ) , ( )] , ( [ ) ( ) ( θ θ θ

θ

a a a

I I

• 1
• 1

l l ∂ = ζ χ ζ

4/15/2010 14

. ) , ( )] , ( [ ) ( ) ( θ θ θ

θ

a a a

I I

l l ∂ ζ χ ζ

SLIDE 15

6 C i t f t f l ti l f ili

• 6. Consistency factors for location-scale families:

( ) ( )

i i d ; σ μ σ μ ζ σ μ | ) , ( | X X x x f x x f

I

=

( ) ( ) ( ) ( ),

, i.i.d. ; σ μ μ ζ σ μ σ μ η σ μ

σ

| ) ( | , , | , ) , ( , | ,

1 , 1 2 1 2 1 2 1 2 1

x f x f X X x x f x x x x f

I I I I I I

= =

( ) ( ) ( ) ( ).

, σ μ σ ζ μ σ σ μ η σ μ

μ σ

, | ) ( , | , | ) ( , |

1 , 1 1 1 , 1

x f x f x f x x f

I I I I I I

=

( ) ( ).

σ μ η μ σ

μ

, | ) ( , |

1 1 , 1

x f x x f

I I I

Invariance of a location scale family: Invariance of a location-scale family:

( )

, , | x x FI ⎟ ⎠ ⎞ ⎜ ⎝ ⎛ − Φ = σ μ σ μ , ) , ( ) , ( )] , ( ), , [( ] ), , [( G b a a b a b a l b x a x b a l = × ∈ ⎭ ⎬ ⎫ + ≡ + ≡ ⎠ ⎝

+ 

 σ μ σ μ σ

4/15/2010 15

) ( )] ( ) [( ⎭ μ μ

SLIDE 16

;

+

∈ ∈ ⎟ ⎞ ⎜ ⎛ − =   b a b b a b a σ μ ζ χ χ σ μ ζ ) ( ) , ( ) ( , ; , ; ∈ − = ∈ ∈ ⎟ ⎠ ⎜ ⎝ =    b b b b a a a b a a

I I I I

μ ζ χ μ ζ ζ χ σ μ ζ

σ σ , , 2

) ( ) ( ) ( , , ) , ( ) , ( . ;

+

∈ ⎟ ⎠ ⎞ ⎜ ⎝ ⎛ =  a a a a

I I

σ ζ χ σ ζ

μ μ , ,

) ( ) (

( ) ( ) ( )

.

• bjective

1 ) ( , ) ( ) , ( , | , , | , , | ,

, 1 , 1 1 2 1

= = = ⇒

−

μ ζ σ σ ζ σ μ ζ μ σ σ μ σ μ

σ μ I I I I I I

x f x f x x f 1. n Propositio

4/15/2010 16

SLIDE 17

Example 3 (Normal family).

) ( 1

2 ⎫

⎧

( )

, 2 ) ( exp 2 1 , |

2 2 2 1 1

x x fI ⎫ ⎧ ⎭ ⎬ ⎫ ⎩ ⎨ ⎧ − − = σ μ σ π σ μ

( )

, 2 ) ( exp 2 , |

2 2 2 2 1 2 1 1

x x x fI ⎫ ⎧ ⎭ ⎬ ⎫ ⎩ ⎨ ⎧ − − − = σ μ σ μ π μ σ

( )

, 2 ) ( ) ( exp , | ,

2 2 2 2 1 3 2 1 2 1

x x x x x x fI ⎫ ⎧ ⎭ ⎬ ⎫ ⎩ ⎨ ⎧ − + − − − = σ μ μ σ π σ μ

( ) ( )

, 4 ) ( exp , | , , |

2 2 2 1 2 2 1 2 1 2 1

x x x x d x x f x x f

I I

⎭ ⎬ ⎫ ⎩ ⎨ ⎧ − − − = = ∫

∞ ∞ −

σ σ π μ σ μ σ

( ) ( )

. ) ( 2 2 1 , | , , |

2 1 2 2 2 2 1 2 1 2 1 2 1

x x x x x x d x x f x x f

I I

+ − + + − = = ∫

∞

μ μ π σ σ μ μ

4/15/2010 17

SLIDE 18

Example 3 (cont’d).

( ) ( )

| |

∫

∞

d f f (

) ( )

) ( 1 )! 1 ( 1 , , | , , , |

2 2 1 2 1 1

− ≡ ′ ′ − = =

∑ ∫

−

x x s s d x x f x x f

n n n n n I n I

σ σ μ μ K K freedom).

• f

degrees with

• n

distributi s (Student' . , 1 ) ( ] ) ( [ )! (

1 2 2 2 3 2

2

− ≡ − + ′ − =

∑ =

n x x n s x s

i n i n n n n

n

μ π

4/15/2010 18

SLIDE 19

Example 4 (Exponential family).

≡ ≡ τ μ ln lnt x

( )

{ }

( )

( ) { }

⇒ ∈ − = = ′ − = ⇒ ≡ ≡

− − ′ − − ′

= =

μ ζ μ μ μ τ μ

σ σ

μ μ μ μ 1 1 1

1 ) ( ; exp | ; exp | ln , ln

1 1 1 1

e e x f I e e x f t x

x x I x x I

 ⎫ ⎧ = ⇒ = ⇒

− ′ =

τ τ ζ μ ζ

σ

1

) ( 1 ) (

1

t t

I I

. ⎭ ⎬ ⎫ ⎩ ⎨ ⎧− = ⇒ τ τ τ

1 2 1 1

exp ) | ( t t t fI

)

1 | 1 = t τ

(

fI τ

τ

4/15/2010 19

SLIDE 20

• 7. Consistency factors in the absence of symmetry:

Example 5 (Binomial vs Normal distribution) Example 5 (Binomial vs. Normal distribution).

≤ ∈ ∈ − ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛ =

−

; ; ) 1 ( ) , | ( θ θ θ n n n n n n n n p

n n n I

  ⎭ ⎬ ⎫ ⎩ ⎨ ⎧ − − = − ≤ ∈ ⎠ ⎝

∫ ∑

+ ∞ − = 2 2 5 .

' 2 ) ' ( exp 2 1 ) , | ( ) , | ( : 1 ) 1 ( , ; σ μ σ π θ θ θ θ dx x n i p n n F n n n n n n

n n i I

> q



 ⎫ ⎧ − ≈ − = = / ) ( ) 1 ( , θ θ σ θ μ n n n n n n F

( )

. ⎭ ⎬ ⎫ ⎩ ⎨ ⎧ − − ≈ ⇒

2 2

2 ) ( exp 2 , | σ μ σ π σ θ n n n fI

I

F

) , | ( n n FI θ ) | ( F

x

) , | (

'

σ μ x FI

4/15/2010 20

x

SLIDE 21

VII Interpretation of Inverse Probability Distributions:

• VII. Interpretation of Inverse Probability Distributions:
• 1. Credible intervals and regions,

2 Degrees of belief and calibration

• 2. Degrees of belief and calibration,
• 3. Fiducial argument,

4 Theorem of Stein Chang and Villegas

• 4. Theorem of Stein, Chang and Villegas,
• 5. Calibration of marginal distributions.

4/15/2010 21

SLIDE 22

• 1. Credible intervals and regions :

Definition 3.a (Credible interval):

( )

⊆ ⇒ ∈ ∃

Θ

V x x f

a I

δ θ θ θ θ : content y probabilit (inverse-) with , interval Credible

b]

( , ; | 

( )

∫

=

b a

d x fI

θ θ

δ θ θ . |

Definition 3.b (Credible region):

( )

∈ ∈ ∃

n m I

f x θ x θ , ; |  

( )

. : content y probabilit with region Credible δ δ = ⊆ ⊆ ⇒

∫

U m I m

d f V U θ x θ x

Θ

| ) ( 

∫

U

4/15/2010 22

SLIDE 23

• 2. Degrees of belief and calibration :

The inferred parameter, although unknown, is usually fixed. What is distributed is our degree of belief that given observed the true value

x

distributed is our degree of belief that, given observed , the true value

• f the inferred parameter is in the credible region .

) (x U x

Definition 4 (Calibration): The inverse probability density is called calibrated if there is at least one algorithm to construct

( )

x θ |

I

f

is called calibrated if there is at least one algorithm to construct credible regions whose (inverse-) probability content is equal to the long-term relative frequency, called coverage, of the regions th t th t l ( ) f th t b th di t ib ti that cover the true value(s) of the parameter, be the distribution

• f the true values in the ensemble what it may.

4/15/2010 23

SLIDE 24

• 3. Fiducial argument.

τ τ τ τ

) [ ]

γ α − ∈ 1 , 1

) ( )

, 5 ∞ ∈ τ

1

τ1 τ1 τ

b

τ

) ) ( ) ) ( )

α τ τ + =

1 1

| 4 | : 3 2

a I a

t F t t F t

) ) ( ) ( )

b a b a

t t t t , , 7 value true ; 6

1 1 1 1

∈ ⇔ ∈ τ τ τ τ

1 1 1 a

τ

) ( ) ( ) ( ) ( )

τ τ τ γ α τ − = ≤ < ⇒ + =

1 1 1 1

| | | Pr | : 4

a I b I b a I b I b

t F t F t t t t F t t t

a

t t

a

t

b

t

a

t

b

t t

1

t

( ) ( )

γ =

1 1

| |

a I b I

( )

⎫

Remark 4.

( ) ( ) ( ) ( ).

| | | |

1 1 1 1 b I a I a I b I

t F t F t F t F τ τ γ γ α τ α τ − = ⇒ ⎭ ⎬ ⎫ + = =

( ) ( ) ( ) ( ) ( )

. | | | | |

1 1 1 1 1

∫

= − = ⇒ ⎭ ⎬ ⎫ + = =

b a

d t f t F t F t F t F

I a I b I b I a I τ τ

τ τ τ τ δ δ α τ α τ

4/15/2010 24

( )

| 1 ⎭

b I

SLIDE 25

admissible all for argument) (Fiducial 2 n Propositio γ α γ δ

( ) ( ) ( ) ( )

and mal infinitesi For . , admissible all for Proof argument). (Fiducial 2 n Propositio t F t f t F t f

I I τ

τ τ γ τ τ δ λ τ τ γ α γ δ ∂ Δ Δ −∂ = ⇒ = | | | |

1 1

฀

( ) ( )

t G l . and , mal infinitesi For 1 R k Proof.

a

t F t f

I a I τ τ τ

τ τ γ τ τ δ λ

=

∂ Δ − = Δ = | |

1 1

. argument. General 1. Remark condition. fiducial the satisfy 1 ) ( , ) (

, 1 ,

= =

−

μ ζ σ σ ζ

σ μ I I

6. Example

4/15/2010 25

SLIDE 26

( ) ( ) and

(Lindley). 3 n Propositio | | τ θ

θ

±∂ = x F x f

I I (

) ( ) ( ) ( ) ( )

. and (Lindley). 3 n Propositio ) ( | : ) ( ), ( | ) ( ) ( | | |

'

μ φ μ θ μ θ η θ ζ θ τ θ

θ

− = ∃ ⇒ = ±∂ y y f x y x f x x f x F x f

I I I I I I I

107.

• 102

pp. , Ser. Soc. Statist Roy. J. (1958), D.Lindley 20 B Proof. ) ( ηI

฀

pp y ( ) y

4/15/2010 26

SLIDE 27

• 4. Theorem of Stein, Chang and Villegas:

( )

element the with coincides , group Lie) (e.g., l topologica a under invariant . Villegas) Chang, (Stein, 4 Theorem ) ( | θ θ x G f

I I

ζ

( )

. regions, credible t equivarian

• n

calibrated

• n

measure Haar invaiant

• right

the

• f

)] ( [ )] , ( [ | x x x θ U a U f G

I

l l = ⇒ 296.

• 289

, Statist. J. Canad. (1986), C.Villegas T.Chang, 14 Proof.

฀

.

• n

measure Haar invariant

• right

the

• f

element the is

+ −

× = =   G

I 1

) , ( σ σ μ ζ 7. Example

4/15/2010 27

SLIDE 28

, rectangles t Equivarian ] , [ ] , [ × = U

b a b a

σ σ μ μ d). (Cont' 7 Example g q , ] , ( ] , [ , ] , [ , ] , ( , ) | ( Pr , ) | ( Pr , ) | ( Pr , ) | ( Pr ] [ ] [

3 2 1 3 2 1

× ≡ × ≡ × −∞ ≡ = = = =

+ +

U U U U U U U

a b a b a a I I I I b a b a

x x x x σ μ μ μ μ μ γ ε β α μ μ   ) ( p . , 2 , ) , , ( 1 1 ; 1 , , , , ] , ( ] , [ , ] , [ , ] , (

1 3 2 1

≥ ≡ ≥ − ≥ ≥ − ≤ ≤ n x x

n a b a b a a

K x γ ε β α ε γ β α μ μ μ μ μ ) (

1 n

4/15/2010 28

SLIDE 29

• 5. Calibration and marginal distributions:

rectangles t Equivarian ] [ ] [ σ σ μ μ U × = d) (Cont' 7 Example , rectangles t Equivarian x x x x ] ( ] [ ] [ ] ( , ) | ( Pr , ) | ( Pr , ) | ( Pr , ) | ( Pr ] , [ ] , [

3 2 1

σ μ μ μ μ μ γ ε β α σ σ μ μ

I I I I b a b a

U U U U U U U U × ≡ × ≡ × ∞ ≡ = = = = × =

+ +

  d). (Cont 7 Example . 1 1 ; 1 , , , , ] , ( ] , [ , ] , [ , ] , (

3 2 1

γ ε β α ε γ β α σ μ μ μ μ μ

a b a b a a

U U U ≥ − ≥ ≥ − ≤ ≤ × ≡ × ≡ × −∞ ≡  

( ) ( )calibrated

, calibrated x x | ] [ 1 | ] , [ μ μ μ β σ σ σ γ β

I b a

f U f U ⇒ × = ⇒ = ⇒ × = ⇒ =

+

 

( )

. calibrated x | ] , [ 1 μ μ μ β

I b a

f U ⇒ × = ⇒ = 

• ne

to

• ne

general under conserved Not 2 Remark ization. reparametr

• ne
• to
• ne

general under conserved Not 2. Remark

4/15/2010 29

SLIDE 30

3.a Kalmanov filter – osnovne predpostavke

{ }

( )

n

 Θ θ θ θ θ

( ) ( ) ( )

{ }

( )

{ }

( )

n j j k k n j j k k

t x x x x t   ∈ = ≡ ∈ = ≡ ; ,..., ; ,...,

1 1

X Θ θ θ θ θ : frekvenčna interpretacija

( ) ( ) ( ) ( )

j j j j j j j j j j

f Q A N f f θ θ θ θ θ θ | , ~ | , |

1 1 1 + + +

= X Θ : frekvenčna interpretacija

( ) ( ) ( ) ( )

j j j j j j j

f V N x f x f θ θ θ | , ~ | , |

1 = −

X Θ

1

x

4

x x , θ

1

x

4

x x , θ

: frekvenčna interpretacija

( )

j j

x f θ |

( ) ( )

V H N f | θ θ

1

x

2

θ

3

x

k

x

1 − k

θ

3

θ

4

θ

k

θ

1

x

2

θ

3

x

k

x

1 − k

θ

3

θ3 θ

4

θ4 θ

k

θ

: obrnljiva

( ) ( )

j j j j j j

H V H N x f , ~ | θ θ

t t t t

k

t

k

t t

2

x

1 − k

x

1

θ t t t t

k

t

k

t t

2

x2 x

1 − k

x

1 − k

x

1

θ

( )

? | =

k k

f X θ

1

t

2

t

3

t

4

t

1 − k

t

k

t t

1

t

2

t

3

t

4

t

1 − k

t

k

t t 4/15/2010 30

SLIDE 31

( ) ( ) ( )

x r r R r N x f

n

; , ? |

1 1 1 1 1 1 1

~

∈ = ∃  θ

( ) ( ) ( )

( )

k j x f x r r R r N x f

j j j

,..., 2 ; | , ; , |

1 1 1 1 1 1 1 1

~

= ∃ ∈ ∃

−

X Θ  θ

( ) ( )

( ) ( )

( ) ( ) ( )

∏

= − −

= ⇒

k j j j j j j j k k

x f f x f x f f

2 1 1 1 1

| | | | | X X Θ θ θ θ θ

( ) ( ) ( )

k k k n n k k k k

R r N d d f f , ~ | |

1 1

∫ ∫

−

= ⇒

 

 

θ θ θ K L X Θ X

( ) ( ) ( )

1 1 1 1 1 1 1 1 1

;

− − − − − − − − −

+ + = ∈ = − + =

T n j j j j j j j j j j j

Q A R A V R r r r A x V R r A r  X

( )

1 1 1 1 − − − −

+ + =

j j j j j j

Q A R A V R

• T. N. Thiele (1880). Om Anvendelse af mindste Kvadraters Methode..., § 2.

P Swerling (1959) J Astronaut Sci 6 str 46 52

• P. Swerling (1959). J. Astronaut. Sci. 6, str. 46-52.
• R. L. Stratonovich (1959). Radiofizika 2:6, str. 892-901.
• (1960). Radio Eng. Electr. Phys. 5:11, str. 1-19.

R E Kalman (1960) Trans ASME J Basic Eng 82 str 34-45

4/15/2010 31

• R. E. Kalman (1960). Trans. ASME J. Basic Eng. 82, str. 34-45.

SLIDE 32

3.b Inicializacija Kalmanovega filtra

( ) ( ) ( ) ( ) ( )

1 1 1 1 1 1 1

| | x x f x f f η θ θ ς θ θ = ⇒ ∃ /

Invarianca glede na translacije in inverzijo +

( )

1 1 |θ

x f

+

• bjektivnost

⇓ ( ) ( ) ( )

| | 1 x f x f ⇒ θ θ θ ς

( )

1

θ ς

Rešitev funkcijskih enačb za : ( ) ( ) ( ) ( ) ( )

1 1 1 1 1 1 1 1 1 1 1 1 1

, ; , ~ | | | 1 V R x r R r N x f x f x f = = ⇒ = ⇒ = θ θ θ θ ς

4/15/2010 32

SLIDE 33

3 I t t ij f(q |X ) 3.c Interpretacija f(qk|Xk):

( ) ( )

f x f X | |

1 1

θ θ ev porazdelit ev porazdelit ≠ ≠ θ θ 1

( )

k k

f X | θ ⇒ ev porazdelit ≠

k

θ

Merljive napovedi: ( ) ( ) ( ) ( ) ( )

k k k k k k k k

f R N r f r f f θ θ θ θ θ | ; , ~ | ; | | = X invariantnost na translacije;

( )

k k

r f θ | : frekvenčna interpretacija.

( )

k

r C = ekvivariantno območje zaupanja:

( )

γ θ θ =

∫

k n k k

d f X | = delež C(rk), ki vsebujejo resnične qk, ne

( ) ( )

. :

n k k

a a r C a r C  ∈ + = +

( )

γ

∫

C k k k

f | ( k), j j

k,

glede na porazdelitev qk v ansamblu.

• C. Stein (1965). Proc. Int. Research Seminar UC Berkeley, str. 217-240.

4/15/2010 33

• T. Chang, C. Villegas (1986). Canad. J. Statist. 14, str. 289-296.

SLIDE 34

Ekvivariantni (n-dim.) pravokotniki

( )

2 k

μ

( )

2 , b k

μ

( )

2 k

θ

( )

1 k

μ

( )

1 ,b k

μ

• (

)

2 k

μ

( )

2 k

μ

( )

2 , b k

μ ( )

2 , b k

μ

( )

2 k

θ

( )

1 k

μ( )

1 k

μ

( )

1 ,b k

μ

( )

1 ,b k

μ

• ( )

( )

( )

( )

( )

( )

( ) ( ) ( ) ( )

n k i k i k k k k k i k n k k k

d d d d r f r f , | | , , ,

1 1 1 1

= =

∫ ∫

+ −

θ θ θ θ θ θ θ θ θ

 

K K L K

, k

μ

( )

1 k

θ φ

• ,

k

μ ,

k

μ

( )

1 k

θ φ

• ( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( ) ( )

i k k i k i k k i k i b k i a k

d r f d r f A

i b k i i a k

| , | : ,

, ,

, ,

= = ≡

∫ ∫ ∞

−

γ θ θ α θ θ θ θ

θ θ θ

( )

2 ,a k

μ

( )

1 ,a k

μ

• ( )

2 ,a k

μ( )

2 ,a k

μ

( )

1 ,a k

μ ( )

1 ,a k

μ

• (

)

( )

( )

k k k k

r A P f

i a k

| |

,

= ⇒

∫

γ γ

θ

≠ delež A(rk), ki vsebujejo resnične .

( )

i k

θ ; , , :

1 k k k k k k k T k k k k T k k

r O s O D O R O O O O ≡ ≡ = = ∃

−

θ μ = diagonalna;

( )

i k

μ

( ) ( )

( )

( ) ( )

( )

( )

k i b k i a k n k k k

s B P B | , , , ,

, , 1

= ⇒ ≡ = γ μ μ μ μ μ K

= delež B(sk), ki vsebujejo resnične

4/15/2010 34

k

μ

resnične .

SLIDE 35

( ) ( ) ( ) ( ).

, | , , | ; , ; , :

1 k k k k k k k k k k k k k k k T k k k k T k k

D s N s f D N s f r O s O D O R O O O O = = ≡ ≡ = = ∃

−

μ μ μ θ μ

Ekvivariantni (n-dim.) pravokotniki

( )

2 k

μ( )

2 k

μ

( )

2

( )

2

( )

2 k

θ

( )

1 k

μ

( )

1 k

μ

( )

1 ,b k

μ( )

1 ,b k

μ

( )

( )

( ) ( ) (

)

( )

( )

( ) ( ) (

)

| , |

1 , 1 ,

2 1 1 2 1

β μ μ μ μ α μ μ μ μ

μ μ

= =

∫ ∫ ∫ ∫ ∫ ∫

∞ ∞ ∞ ∞ − ∞ ∞ − ∞ − n k k n k k k

s f d d d s f d d d

b k a k

L L

μ μ

( )

2 , b k

μ ( )

2 , b k

μ

( )

1

θ

, k

μ ,

k

μ φ

( )

( )

( ) ( ) (

)

( )

( ) ( )

( )

( ) ( )

( )

( ) ( )

( )

; | , |

, 2 , 2 1 , 1 1 ,

2 1 1

γ β μ μ μ μ β μ μ μ μ

μ μ μ μ

= = =

∫ ∫ ∫ ∫ ∫ ∫

∞ − ∞ − n k k n k k k k k k k k

s f d d d s f d d d

n b k b k b k a k

L M L

( )

2 ,a k

μ( )

2 ,a k

μ

( )

k

θ

( )

1 a k

μ ( )

1 a k

μ

( ) ( ) ( )

( )

; |

, 2 , 1 ,

γ β μ μ μ μ

μ μ μ

∫ ∫ ∫

n k k k k k

f

n a k a k a k

( ) ( )

( )

( ) ( )

( );

, , , , ,

1 1 n k k k n k k k

μ μ μ θ θ θ K K = =

, a k

μ

, a k

μ

[ ]

. 1 1 , 1 , ,

2 2 1 1

γ β α β α β α ≥ ≥ ≥ − ≥ ≥ − ∈ K

j j

( )

( )

( )

( ) ( ) ( ) ( )

∫ ∫

+ −

= ⇒ = = =

n j j j

d d d d s f s f μ μ μ μ μ μ γ β β α L

1 1 1

| | 1

kalibrirane

( )

( )

∫ ∫

− −

= ⇒ = = =

  k k k k k k k k j j j

d d d d s f s f μ μ μ μ μ μ γ β β α K K L

1 1

| | , 1 ,

kalibrirane

( )

( )

( )

( ) ( ) ( ) ( )

∫ ∫

+ −

=

  n k j k j k k k k k j k

d d d d s f s f θ θ θ θ θ θ K K L

1 1 1

| |

niso kalibrirane

4/15/2010 35

SLIDE 36

3.b Inicializacija Kalmanovega filtra

( ) ( ) ( ) ( ) ( )

1 1 1 1 1 1 1

| | x x f x f f η θ θ ς θ θ = ⇒ ∃ /

( ) ( )

; θ θ ∈ + = + = a a a l a x x a l 

Invarianca :

( )

1 1 |θ

x f

( ) ( ) ( ) ( )

1 1 2 1 1 2 1 1 1 1 1 1

, ; , , , θ θ θ θ − = − = ∈ + = + = l x x l a a a l a x x a l 

( ) ( ) ( )

| | 1 x f x f ⇒ θ θ θ ς

( )

1

θ ς

Rešitev funkcijskih enačb za : ( ) ( ) ( ) ( ) ( )

1 1 1 1 1 1 1 1 1 1 1 1 1

, ; , ~ | | | 1 V R x r R r N x f x f x f = = ⇒ = ⇒ = θ θ θ θ ς

4/15/2010 36

SLIDE 37

Text box With shadow

4/15/2010 37