Poisson Disorder Problems Model A Statistic Solution Examples - PowerPoint PPT Presentation

Poisson Disorder Problems Savas Dayanik • • • Tweedie New Researcher Invited Lecture Problem Poisson Disorder Problems Model A Statistic Solution Examples Savas Dayanik Appendix Princeton University References ⊳ ◭ ◮ ⊲ Page 1 of 16 Ninth Meeting of New Researchers in Statistics and Go Back Probability • Seattle, August 1-5, 2006 Close

1. Problem Description Poisson Disorder Problems Let X be a compound Poisson process whose rate λ 0 and Savas Dayanik jump distribution ν 0 ( · ) change to λ 1 and ν 1 ( · ) , respectively, at • • • Problem some unknown and unobservable time θ . Model Jump distr. ν 0 A Statistic Rate λ 0 Solution Jump distr. ν 1 Rate λ 1 Examples Appendix References Process X θ 0 ⊳ ◭ ◮ ⊲ (Disorder time) Page 2 of 16 Go Back Close

1. Problem Description Poisson Disorder Problems Let X be a compound Poisson process whose rate λ 0 and Savas Dayanik jump distribution ν 0 ( · ) change to λ 1 and ν 1 ( · ) , respectively, at • • • Problem some unknown and unobservable time θ . Model A Statistic Solution (Only the process X Examples is observable.) Appendix References Process X 0 ⊳ ◭ ◮ ⊲ Page 2 of 16 Go Back Close

1. Problem Description Poisson Disorder Problems Let X be a compound Poisson process whose rate λ 0 and Savas Dayanik jump distribution ν 0 ( · ) change to λ 1 and ν 1 ( · ) , respectively, at • • • Problem some unknown and unobservable time θ . Model A Statistic Solution (Only the process X Examples is observable.) Appendix References Process X 0 ⊳ ◭ ◮ ⊲ Page 2 of 16 Problem: Find a decision rule which Go Back • detects the disorder time θ as quickly as possible, Close • is adapted to the history of X .

Poisson Let (Ω , F , P ) be a probability space supporting random vari- Disorder Problems ables θ , Y 1 , Y 2 , · · · , a counting process N = { N t ; t ≥ 0 } . Define Savas Dayanik � • • • N t � X t = X 0 + Y k ≡ X 0 + (0 ,t ] × R d y p ( ds, dy ) , t ≥ 0 Problem k =1 Model in terms of the point process describing jump times and sizes A Statistic ∞ � p ((0 , t ] × A ) � t ≥ 0 , A ∈ B ( R d ) . 1 { σ k ≤ t } 1 { Y k ∈ A } , Solution k =1 Examples and σ k = inf { t > σ k − 1 : X t � = X t − } , k = 1 , 2 , . . . ( σ 0 ≡ 0 ). Appendix as the natural filtration of X, F = {F t } t ≥ 0 (1) References G t � F t ∨ σ { θ } . G = {G t } t ≥ 0 , The disorder time θ has the distribution ⊳ ◭ ◮ ⊲ P { θ > t | θ > 0 } = e − λt , (2) and P { θ = 0 } = π t ≥ 0 . Page 3 of 16 The counting process { p ( t, A ) � p ((0 , t ] × A ); t ≥ 0 } is a non- Go Back homogeneous Poisson process with the ( P , G )-intensity Close h ( t, A ) � λ 0 ν 0 ( A )1 { t<θ } + λ 1 ν 1 ( A )1 { t ≥ θ } , (3) t ≥ 0 .

Poisson Our problem is (i) to calculate the minimum Bayes risk Disorder Problems V ( π ) � inf τ ∈ F R τ ( π ) , Savas Dayanik (4) • • • � ( τ − θ ) + � R τ ( π ) � P { τ < θ } + c · E , π ∈ [0 , 1) , Problem and (ii) to find an F -stopping time τ where the infimum is Model attained (if exists, called a minimum Bayes detection rule ). A Statistic The Bayes risk R τ ( π ) in (4) associated with every F -stopping Solution time τ is the sum of Examples • the false alarm frequency P { τ < θ } , and Appendix References • the expected detection delay cost c · E [( τ − θ ) + ] . ⊳ ◭ ◮ ⊲ Page 4 of 16 Go Back Close

Poisson Our problem is (i) to calculate the minimum Bayes risk Disorder Problems V ( π ) � inf τ ∈ F R τ ( π ) , Savas Dayanik (4) • • • � ( τ − θ ) + � R τ ( π ) � P { τ < θ } + c · E , π ∈ [0 , 1) , Problem and (ii) to find an F -stopping time τ where the infimum is Model attained (if exists, called a minimum Bayes detection rule ). A Statistic The Bayes risk R τ ( π ) in (4) associated with every F -stopping Solution time τ is the sum of Examples • the false alarm frequency P { τ < θ } , and Appendix References • the expected detection delay cost c · E [( τ − θ ) + ] . Standard Bayes risks include ⊳ ◭ ◮ ⊲ R τ ( π ) = P { τ < θ } + c E [( τ − θ ) + ] , Linear delay penalty: Page 4 of 16 R ( ε ) τ ( π ) � P { τ < θ − ε } + c E [( τ − θ ) + ] , Go Back Expected miss: R ( miss ) ( π ) � E [( θ − τ ) + ] + c E [( τ − θ ) + ] , τ Close ( π ) � P { τ < θ } + c E [ e α ( τ − θ ) + − 1] . R ( exp ) Expon. delay penalty: τ

Where do the disorder problems arise? Poisson Disorder Problems Insurance companies: Recalculate the premiums for the fu- Savas Dayanik • • • ture sales of insurance policies when the risk structure Problem changes (e.g., the arrival rate of claims of certain size). Model Airlines, retailers of perishable products: Adjust the prices A Statistic when a change in the demand structure is detected (e.g., Solution the arrival rate of a certain type of customers). Examples Appendix Quality control and maintenance: Inspect, recalibrate, or References repair tools and machines as soon as a manufacturing process goes out of control. ⊳ ◭ ◮ ⊲ Fraud and computer intrusion detection: Alert the inspec- Page 5 of 16 tors for an immediate investigation as soon as abnormal Go Back credit card activity, cell phone calls, or computer network traffic are detected. Close

2. The Model Poisson Disorder Problems Let (Ω , F , P 0 ) be a p.s. with independent random elements: Savas Dayanik • a Poisson process N = { N t ; t ≥ 0 } with rate λ 0 , • • • Problem • iid R d -valued rv’s Y 1 , Y 2 , . . . with distr. ν 0 ( · ) ( ν 0 ( { 0 } ) = 0 ), Model • a rv θ with the distribution A Statistic P 0 { θ > 0 } = (1 − π ) e − λt , t ≥ 0 . and P 0 { θ = 0 } = π Solution A compound Poisson process with arrival rate λ 0 and jump Examples distribution ν 0 ( · ) is defined by Appendix � N t � X t = X 0 + Y k = X 0 + y p ( ds, dy ) , t ≥ 0 References (0 ,t ] × A k =1 in terms of the point process on ( R + × R d , B ( R + ) × B ( R d )) ⊳ ◭ ◮ ⊲ ∞ � p ((0 , t ] × A ) � t ≥ 0 , A ∈ B ( R d ) . 1 { σ k ≤ t } 1 A ( Y k ) , Page 6 of 16 k =1 Under P 0 the process { p ((0 , t ] × A ); t ≥ 0 } is homogeneous Go Back Poisson process with the F -intensity λ 0 · ν 0 ( A ) . Each σ k is a Close jump time of X , and F is its history, and G = F ∨ σ { θ } .

Poisson Let λ 1 be a constant, and ν 1 ( · ) be a probability measure on Disorder Problems ( R d , B ( R d )) absolutely continuous wrt ν 0 ( · ) with RN-derivative Savas Dayanik • • • f ( y ) � dν 1 y ∈ R d . ( y ) , Problem dν 0 Define locally a new probability measure P on (Ω , ∨ t ≥ 0 G t ) by Model the Radon-Nikodym derivatives A Statistic � � λ 1 � N t � � Solution d P � = 1 { t<θ } + 1 { t ≥ θ } e − ( λ 1 − λ 0 )( t − θ ) (5) f ( Y k ) , t ≥ 0 . � d P 0 λ 0 Examples G t k = N θ − +1 Appendix Then every counting process { p ((0 , t ] × A ); t ≥ 0 } , A ∈ B ( R d ) is References a nonhomogeneous Poisson process with the ( P , G ) -intensity (3) h ( t, A ) = λ 0 ν 0 ( A )1 { t<θ } + λ 1 ν 1 ( A )1 { t ≥ θ } . ⊳ ◭ ◮ ⊲ Since P 0 ≡ P on G 0 = σ { θ } , the disorder time θ has the same Page 7 of 16 distribution under P 0 and P . Go Back Therefore, the model under the measure P of (5) has the Close same setup described in the beginning.

3. A Markovian sufficient statistic for detection problem Poisson Disorder Problems Savas Dayanik The Bayes risk R τ ( π ) = P { τ < θ } + E [( τ − θ ) + ] , π ∈ [0 , 1) in (4) • • • for every F -stopping rule τ can be written as Problem �� τ � � � Φ t − λ Model e − λt (6) R τ ( π ) = 1 − π + c (1 − π ) E 0 dt . c 0 A Statistic The expectation in (6) is taken under the ref. p.m. P 0 , and Solution Φ t � P { θ ≤ t |F t } (7) P { θ > t |F t } , t ∈ R + . Examples The process Φ is a piecewise-deterministic Markov process: Appendix   � �   References Φ t = x t − σ n − 1 , Φ σ n − 1 , t ∈ [ σ n − 1 , σ n )   , n ≥ 1 . Φ σ n = λ 1    f ( Y n )Φ σ n −  λ 0 ⊳ ◭ ◮ ⊲ The fucntion x ( · , φ ) = { x ( t, φ ); t ≥ 0 } is the solution of Page 8 of 16 d dtx ( t, φ ) = λ + ax ( t, φ ) , t ∈ R , and x (0 , φ ) = φ ; i.e. , Go Back x ( t, φ ) = φ d + e at [ φ − φ d ] , t ∈ R . Close Here a � λ − λ 1 + λ 0 , φ d � − λ/a .

Poisson The min. Bayes risk in (4) of the Poisson disorder problem is Disorder � � Problems π U ( π ) = 1 − π + c (1 − π ) · V , π ∈ [0 , 1) . Savas Dayanik 1 − π • • • The function V : R + �→ ( −∞ , 0] is the value function of the Problem discounted optimal stopping problem Model � �� τ � � � V ( φ ) � inf e − λt g (Φ t ) dt A Statistic (8) � Φ 0 = φ τ ∈ F E 0 0 Solution t t with the running cost Examples Φ t ( ω ) function Appendix g ( φ ) � φ − λ c, φ ≥ 0 . References Φ t ( ω ) for the piecewise deter- ⊳ ◭ ◮ ⊲ ministic Markov process Page 9 of 16 Φ . Go Back [ Left: sample paths of Close 0 λ/c φ d φ 0 λ/c φ (a) φ d > 0 (b) φ d < 0 the process Φ ]