Anderson-Darling Type Goodness-of-fit Statistic Based on a Multifold - PowerPoint PPT Presentation

Anderson-Darling Type Goodness-of-fit Statistic Based on a Multifold Integrated Empirical Distribution Function S. Kuriki (Inst. Stat. Math., Tokyo) and H.-K. Hwang (Academia Sinica) Bernoulli Society Satellite Meeting to ISI2013 Wed 4 Sept 2013, Univ Tokyo 1 / 27

Outline I. Anderson-Darling statistic and its extension II. Limiting null distribution III. Moment generating function IV. Statistical power V. Extension of Watson’s statistic Summary 2 / 27

I. Anderson-Darling statistic and its extension 3 / 27

Goodness-of-fit tests ▶ X 1 , . . . , X n : i.i.d. sequence from cdf F ▶ Goodness-of-fit test: H 0 : F = G vs. H 1 : F ̸ = G ( G is a given cdf) ▶ When G is continuous, we can assume G ( x ) = x (i.e., Unif(0,1)) WLOG. ▶ Empirical distribution function ∑ n F n ( x ) = 1 1 l( X i ≤ x ) n i =1 ▶ Test statistic is defined as a measure of discrepancy between F n ( x ) and G ( x ) = x . 4 / 27

Goodness-of-fit tests (cont) ▶ We focus on two integral-type test statistics. ▶ Anderson-Darling (1952) statistic: ∫ 1 1 x (1 − x )( F n ( x ) − x ) 2 dx A n = n 0 ▶ Watson (1961) statistic (for testing uniformity on the unit sphere in R 2 ): ∫ 1 {∫ 1 } 2 ( F n ( x ) − x ) 2 dx − n U n = n ( F n ( x ) − x ) dx 0 0 ▶ Limiting null distributions: Let ξ 1 , ξ 2 , . . . be i.i.d. sequence from N (0 , 1). As n → ∞ , ∞ ∞ ∑ ∑ 1 1 d d k ( k + 1) ξ 2 2 π 2 k 2 ( ξ 2 2 k − 1 + ξ 2 A n → k , U n → 2 k ) k =1 k =1 5 / 27

Closely looking at Anderson-Darling ▶ Anderson-Darling statistic ∫ 1 B n ( x ) 2 where B n ( x ) = √ n ( F n ( x ) − x ) A n = x (1 − x ) dx 0 Here, ∫ 1 B n ( x ) = √ n h [0] ( t ; x ) dF n ( t ) , h [0] ( t ; x ) = 1 l( t ≤ x ) − x 0 ▶ h [0] ( t ; x ) t 0 x 1 6 / 27

An extension to Anderson-Darling ▶ To propose a new class of test statistics, instead of h [0] ( · ; x ), we prepare different type h [ m ] ( · ; x ). ▶ Note first that h [0] ( · ; x ) is piecewise constant s.t. ∫ 1 0 h [0] ( t ; x ) · 1 dt = 0 ▶ Define h [1] ( · ; x ) to be continuous and piecewise linear s.t. ∫ 1 0 h [1] ( t ; x ) · ( at + b ) dt = 0 , ∀ a , b ▶ h [1] ( · ; x ) t 0 x 1 7 / 27

An extension to Anderson-Darling (cont) ▶ General from of h [ m ] ( t ; x ): h [ m ] ( t ; x ) = 1 m !( x − t ) m 1 l( t ≤ x ) ∫ x ∑ m 1 m !( x − u ) m L k ( u ) du × L k ( t ) − 0 k =0 where L k ( · ) is the Legendre polynomial of degree k ▶ h [2] ( · ; x ) t 0 x 1 8 / 27

An extension to Anderson-Darling (cont) ▶ We propose an extension of Anderson-Darling: ∫ 1 B [ m ] ( x ) 2 A [ m ] n = { x (1 − x ) } m +1 dx n 0 where ∫ 1 ( x ) = √ n B [ m ] h [ m ] ( t ; x ) dF n ( t ) n 0 { ∫ ∫ = √ n · · · F n ( x 1 ) dx 1 · · · dx m x > x m > ··· > x 1 > 0 } ∫ ∫ ∫ 1 m ∑ − · · · L k ( x 1 ) dx 1 · · · dx m L k ( t ) dF n ( t ) 0 k =0 x > x m > ··· > x 1 > 0 ( m -fold integral of empirical distribution function) ▶ A [ m ] is well-defined (the integral exists) whenever X i ∈ (0 , 1). n ▶ A [0] is the original Anderson-Darling. n 9 / 27

II. Limiting null distribution 10 / 27

Main results — Limiting null distribution ▶ W ( · ) : the Winer process on [0 , 1] B ( x ) = W ( x ) − xW (1) : Brownian bridge ∫ 1 ▶ Let B [ m ] 0 h [ m ] ( t ; x ) dB n ( x ). ( x ) = n We can prove that as n → ∞ , ∫ 1 B [ m ] d → B [ m ] ( · ) in L 2 , where B [ m ] ( x ) = h [ m ] ( t ; x ) dB ( t ) ( · ) n 0 and hence (by continuous mapping) ∫ 1 ∫ 1 B [ m ] ( x ) 2 B [ m ] ( x ) 2 → A [ m ] := A [ m ] n d = { x (1 − x ) } m +1 dx { x (1 − x ) } m +1 dx n 0 0 ▶ We will examine these limiting distributions B [ m ] ( · ) and A [ m ] . 11 / 27

Main results — Limiting null distribution (cont) Theorem (Karhunen-Lo` eve expansion) √ ∑ ∞ B [ m ] ( x ) ( k − m − 1)! ( k + m + 1)! L ( m +1) { x (1 − x ) } ( m +1) / 2 = ( x ) ξ k k k = m +1 (uniformly in x, with prob. 1), where ∫ 1 ξ k = L k ( t ) dB ( t ) , i.i.d. N (0 , 1) 0 L ( m +1) is the associate Legendre function. □ k Corollary (Limiting null distribution of A [ m ] n ) ∫ 1 ∑ ∞ B [ m ] ( x ) 2 ( k − m − 1)! A [ m ] = ( k + m + 1)! ξ 2 { x (1 − x ) } m +1 dx = k 0 k = m +1 ξ 2 k ∼ χ 2 (1) i.i.d. □ 12 / 27

III. Moment generating function 13 / 27

Moment generating function ▶ The moment generating function (Laplace transform) of ∞ ∑ 1 A [ m ] = ξ 2 k , λ k = k ( k + 1) · · · ( k + 2 m + 1) λ k k =1 is ( ) − 1 [ e sA [ m ] ] ∞ ∏ 1 − 2 s 2 E = λ k k =1 Theorem Let x j ( s ) ( j = 0 , 1 , . . . , 2 m + 1) be the solution of λ x − 2 s = 0 , i.e., x ( x + 1) · · · ( x + 2 m + 1) − 2 s = 0 Then √ [ e sA [ m ] ] 2 m +1 ∏ Γ(1 − x j ( s )) E = j ! j =0 □ 14 / 27

Moment generating function ( m = 0 ) ▶ When m = 0 (Anderson-Darling), λ k = k ( k + 1) The equation x ( x + 1) − 2 s = 0 has a solution √ x 0 ( s ) , x 1 ( s ) = − 1 2 ± 1 + 8 s ▶ Hence, [ e sA [0] ] √ E = Γ(1 − x 0 ( s ))Γ(1 − x 1 ( s )) √ 2 π s √ 1 + 8 s = − cos π 2 (Anderson and Darling, 1952) ▶ Euler’s reflection formula Γ( z )Γ(1 − z ) = π/ sin( π z ) is used. 15 / 27

Moment generating function ( m = 1 ) ▶ When m = 1. λ k = k ( k + 1)( k + 2)( k + 3). ▶ The equation x ( x + 1)( x + 2)( x + 3) − 2 s = 0 has the explicit solution , because by letting x = y − 3 / 2, LHS =( y − 3 / 2)( y − 1 / 2)( y + 1 / 2)( y + 3 / 2) − 2 s { y 2 − (3 / 2) 2 }{ y 2 − (1 / 2) 2 } = − 2 s is a quadratic equation in y 2 = ( x + 3 / 2) 2 . ▶ As a result, √ ( ) √ x 0 ( s ) , x 1 ( s ) , x 2 ( s ) , x 3 ( s ) = 1 ± 5 ± 4 2 s + 1 − 3 , 2 [ e sA [1] ] π s √ E = √ √ 5 − 4 √ 1 + 2 s ) cos( π 5 + 4 √ 1 + 2 s ) 3 cos( π 2 2 16 / 27

Moment generating function ( m = 2 ) ▶ When m = 2, [ e sA [2] ] ( π s ) 3 / 2 E = √ − 4320 cos( π √ η 1 ) cosh( π √ η 2 ) cosh( π √ η 3 ) where √ √ 3 81 s 2 + 480 s − 1728 η = 27 s + 80 + 3 and ( ) 1 4 η 2 + 35 η + 112 η 1 = 12 η ( 4 e − π i / 3 η 2 − 35 η + 112 e π i / 3 ) 1 η 2 = 12 η ( 4 e π i / 3 η 2 − 35 η + 112 e − π i / 3 ) 1 η 3 = 12 η 17 / 27

Calculation of upper prob. ▶ Finite representation is useful in numerical calculation. ∫ λ 2 k / 2 ∞ ∑ ( ) ( − 1) k − 1 e − xs A [ m ] > x P = √� ds [ e sA [ m ] ]� π � � λ 2 k − 1 / 2 k =1 s � E � (Smirnov-Slepian technique, see Slepian (1958)). ▶ The case m = 0: 1 Upper Prob 0.8 0.6 0.4 0.2 0.5 1 1.5 2 2.5 3 x 18 / 27

IV. Statistical power 19 / 27

Statistical power ▶ We have the sample counterpart of the KL-expansion: √ B [ m ] ∑ ( x ) ( k − m − 1)! n ( k + m + 1)! L ( m +1) ( x ) � { x (1 − x ) } ( m +1) / 2 = ξ k k k ≥ m +1 where ∫ 1 ∑ n 1 � √ n ξ k = L k ( x ) dB n ( x ) = L k ( X i ) , k ≥ 1 0 i =1 ▶ The (extended) Anderson-Darling statistics are also written in terms of � ξ k ’s as ∑ 1 k = 1 1 + 1 2 + 1 A [0] � � � � ξ 2 ξ 2 ξ 2 ξ 2 n = 3 + · · · k ( k + 1) 2 6 12 k ≥ 1 ∑ 1 k = 1 1 A [1] ξ 2 � ξ 2 � ξ 2 � n = 2 + 3 + · · · ( k − 1) k ( k + 1)( k + 2) 24 120 k ≥ 2 20 / 27

Statistical power (cont) ▶ First two components: √ √ � � ξ 1 = 12 nm 1 , ξ 2 = 6 5 n × ( m 2 − 1 / 12) , ∑ n i =1 ( X i − 1 / 2) k (the sample k th moment where m k = 1 n around 1 / 2) ▶ A [0] n = � ξ 2 1 / 2 + · · · has much power for mean-shift alternative, and ▶ A [1] n = � ξ 2 2 / 24 + · · · has much power for dispersion-change alternative 21 / 27

V. Extension of Watson’s statistic 22 / 27

Watson’s statistic and its extension ▶ Similar extensions are possible for Watson’s statistic. Let ∫ 1 ∫ 1 U [ m ] C [ m ] C [ m ] ( x ) 2 dx , h [ m ] ( t ; x ) dF n ( t ) , = = n n n 0 0 where h [ m ] ( t ; x ) = ( t − x ) m 1 1 l( t − x ≤ 0) + ( m + 1)! b m +1 ( t − x ) m ! ▶ b m ( y ) is the Bernoulli polynomial, which satisfies b m ( y + 1) = b m ( y ) + my m − 1 . ▶ U [0] is the original Watson statistic. n ▶ U [1] is proposed by Henze and Nikitin (2002). n 23 / 27

Watson’s statistic and its extension (cont) ▶ h [0] ( · ; x ) 1-x t 0 x 1 ▶ h [1] ( · ; x ) and h [2] ( · ; x ) 1 t 0 x 1 t 0 x 24 / 27

Limiting null distribution ▶ Let C [ m ] ( · ) d U [ m ] d → C [ m ] ( · ) → U [ m ] . and n n ▶ KL-expansion of C [ m ] ( x ): { } ∑ ∞ 1 l [ m ] 2 k − 1 ( x ) ξ 2 k − 1 + l [ m ] C [ m ] ( x ) = 2 k − 1 ( x ) ξ 2 k , (2 k π ) m +1 k =1 where ( ) ( ) 2 k π x − m + 1 2 k π x − m + 1 l [ m ] l [ m ] 2 k − 1 ( x ) = sin π , 2 k ( x ) = cos π , 2 2 ∫ 1 ∫ 1 ξ 2 k − 1 = sin(2 k π x ) dB ( x ) , ξ 2 k = cos(2 k π x ) dB ( x ) . 0 0 ▶ Consequently, ∑ ∞ 1 U [ m ] = (2 k π ) 2( m +1) { ξ 2 2 k − 1 + ξ 2 2 k } . k =1 25 / 27