Renewal Processes Renewal Processes Today: Renewal phenomena Bo - PowerPoint PPT Presentation

Renewal Processes Renewal Processes Today: ◮ Renewal phenomena Bo Friis Nielsen 1 Next week ◮ Markov Decision Processes 1 DTU Informatics Three weeks from now ◮ Brownian Motion 02407 Stochastic Processes 8, October 27 2020 Bo Friis Nielsen Renewal Processes Bo Friis Nielsen Renewal Processes A Poisson proces Underlying Markov Jump Process Let J i ( t ) be the (absorbing) Markov Jump Process related to X i . Define J ( t ) = J i ( t − � N ( t ) Sequence X i , where X i ∼ exp( λ i ) or X i ∼ PH (( 1 ) , [ − λ ]) . j = 1 τ i ) W n = � n i = 1 X i P ( J ( t + ∆) = j | J ( t ) = i ) = S ij + s i α j �� � N ( t ) = max n ≥ 0 { W n ≤ t } = max X i ≤ t Such that n ≥ 0 A = S + s α i = n is the generator for the continued phase proces - J ( t ) Note the Let us consider a sequence, where X i ∼ PH ( α , S ) . similarity with the expression for a sum of two phase-type distributed variables Bo Friis Nielsen Renewal Processes Bo Friis Nielsen Renewal Processes

Distribution of N ( t ) Renewal function For X i ∼ exp( λ ) P ( N ( t ) = n ) = ( λ t ) n n ! e − λ t What if X i ∼ PH ( α , S ) X i ∼ exp( λ ) then M ( t ) = E ( N ( t )) = λ t Probability of having a Generator up to finite n point in [ t ; t + d t ( P ( ∃ n : W n ∈ [ t ; t + d t () The probability of a   point is the probability that J ( t ) has an instantaneous visit to an S s α 0 0 . . . 0 0 absorbing state ( J ( t ) shifts from some J n () to J n + 1 0 S s α 0 . . . 0 0     0 0 S s α . . . 0 0   P ( N ( t + d t ) − N ( t ) = 1 | J ( t ) = i ) = s i d t + o ( d t )   0 0 0 S . . . 0 0 A n =    .. . .. .. . .. . .. .. . .. .. ... ... ... . .. .. .. . ..  α e A t 1 i P ( J ( t ) = i ) = . . . . . . . . . . . . . . . . . .   . . . . . .   α e A t s d t + o ( d t ) P ( N ( t + d t ) − N ( t ) = 1 ) =   0 0 0 0 . . . S s α   � t � t 0 0 0 0 . . . 0 S α e A u s d u = α e A u d u s M ( t ) = E ( N ( t ) = 0 0 A quasi-birth process - to calculate P ( N ( t ) = n ) we would need the matrix-exponential of an ( n + 1 ) p dimensional square matrix The generator A is singular ( A 1 = 0 , π A = 0 ) P ( N ( t ) > n ) = P ( W n ≤ t ) , W n is an “Erlang-type” PH variable Bo Friis Nielsen Renewal Processes Bo Friis Nielsen Renewal Processes � t 0 e A u d u Back to M ( t ) Calculation of First we note that 1 π − A is non-singular � t � t ( 1 π − A ) − 1 ( 1 π − A ) e A u d u e A u d u = 0 0 �� t � t � We have ( 1 π − A ) 1 = 1 and π ( 1 π − A ) = π , so ( 1 π − A ) − 1 1 π e A u d u − A e A u d u = 0 0 α ( 1 π − A ) − 1 � � e A t − I �� M ( t ) = t 1 π − s � t � t � t π s t + α ( 1 π − A ) − 1 s − α ( 1 π − A ) − 1 e A t s ∞ ( A u ) n ∞ π ( A u ) n = 1 π e A u d u � � = 1 π d u = 1 d u n ! n ! 0 0 0 We have α ( 1 π − A ) − 1 e A t s → α ( 1 π − A ) − 1 1 π s = π s n = 0 n = 0 � t = 1 π d u = t 1 π 0 � t e A t − I A e A u d u = 0 Bo Friis Nielsen Renewal Processes Bo Friis Nielsen Renewal Processes

Renewal Processes Age, Residual Life, and Total Life (Spread) F ( x ) = P { X ≤ x } γ t = W N ( t )+ 1 − t ( excess or residual life time ) W n = X 1 + · · · + X n δ t = t − W N ( t ) ( current life or age ) N ( t ) = max { n : W n ≤ t } β t = δ t + γ t ( total life or spread E ( N ( t )) = M ( t ) Renewal function Bo Friis Nielsen Renewal Processes Bo Friis Nielsen Renewal Processes Topics in renewal theory Continuous Renewal Theory (7.6) Elementary renewal theorem M ( t ) → 1 t µ � � W N ( t )+ 1 = µ ( 1 + M ( t )) E � t P { W n ≤ x } = F n ( x ) with M ( t ) = � ∞ n = 1 F n ( t ) , F n ( t ) = 0 F n − 1 ( t − x ) d F ( t ) � x � t F n ( x ) = F n − 1 ( x − y ) d F ( y ) Renewal equation A ( t ) = a ( t ) + 0 A ( t − u ) d F ( u ) 0 Solution to renewal equation The expression for F n ( x ) is generally quite complicated � t � ∞ 0 a ( t − u ) d M ( u ) → 1 A ( t ) = 0 a ( t ) d t Renewal equation µ Limiting distribution of residual life time � x � x lim t →∞ P { γ t ≤ x } = 1 v ( x ) = a ( x ) + v ( x − u ) d F ( u ) 0 ( 1 − F ( u )) d u µ 0 Limiting distribution of joint distribution of age and residual life � x � ∞ time lim t →∞ P { γ t ≥ x , δ t ≥ y } = 1 x + y ( 1 − F ( z )) d z v ( x ) = a ( x − u ) d M ( u ) µ 0 Limitng distribution of total life time (spread) � x 0 t d F ( t ) lim t →∞ P { β t ≤ x } = µ Bo Friis Nielsen Renewal Processes Bo Friis Nielsen Renewal Processes

Expression for M ( t ) E [ W N ( t )+ 1 ]     N ( t )+ 1 N ( t )+ 1 � �  = E [ X 1 ] + E P { N ( t ) ≥ k } = P { W k ≤ t } = F k ( t ) E [ W N ( t )+ 1 ] = X j X j E    j = 1 j = 1 P { N ( t ) = k } = P { W k ≤ t , W k + 1 > t } = F k ( t ) − F k + 1 ( t ) ∞ ∞ � � � � = µ + E X j 1 ( X 1 + · · · + X j − 1 ≤ t ) M ( t ) = E ( N ( t )) = k P { N ( t ) = k } j = 2 k = 1 ∞ ∞ � � X j and 1 ( X 1 + · · · + X j − 1 ≤ t ) are independent = P { N ( t ) > k } = P { N ( t ) ≥ k } k = 0 k = 1 ∞ ∞ ∞ � � � � � � = µ + X j 1 ( X 1 + · · · + X j − 1 ≤ t ) = µ + µ F j − 1 ( t ) E E � = F k ( t ) j = 2 j = 2 k = 1 E [ W N ( t )+ 1 ] = exp[ X 1 ] E [ N ( t ) + 1 ] = µ ( M ( t ) + 1 ) Bo Friis Nielsen Renewal Processes Bo Friis Nielsen Renewal Processes Poisson Process as a Renewal Process The Elementary Renewal Theorem M ( t ) E [ N ( t )] = 1 lim = lim µ t t t →∞ t →∞ n ∞ ( λ x ) n ( λ x ) n e − λ x = 1 − � � e − λ x F n ( x ) = The constant in the linear asymptote n ! n ! i = n i = 0 = σ 2 − µ 2 M ( t ) − µ M ( t ) = λ t � � lim 2 µ 2 t t →∞ Excess life, Current life, mean total life Example with gamma distribution Page 367 Bo Friis Nielsen Renewal Processes Bo Friis Nielsen Renewal Processes

Asymptotic Distribution of N ( t ) Limiting Distribution of Age and Excess Life When E [ X k ] = µ and V ar [ X k ] = σ 2 both finite � x t →∞ P { γ t ≤ x } = 1 lim ( 1 − F ( y )) d y = H ( x ) � x µ � � 0 N ( t ) − t /µ 1 e − y 2 / 2 d y √ t →∞ P lim t σ 2 /µ 3 ≤ x = � ∞ P { γ t > x , δ t > y } = 1 � 2 π −∞ ( 1 − F ( z )) d z µ x + y Bo Friis Nielsen Renewal Processes Bo Friis Nielsen Renewal Processes Size biased distributions Delayed Renewal Process f i ( t ) = t i f ( t ) Distribution of X 1 different � X i � E The limiting distribution og β ( t ) Bo Friis Nielsen Renewal Processes Bo Friis Nielsen Renewal Processes

Stationary Renewal Process Additional Reading S. Karlin, H. M. Taylor: “A First Course in Stochastic Processes” Chapter 5 pp.167-228 Delayed renewal distribution where William Feller: “An introduction to probability theory and its P { X 1 ≤ x } = G s ( x ) = µ − 1 � x 0 ( 1 − F ( y )) d y applications. Vol. II.” Chapter XI pp. 346-371 Ronald W. Wolff: “Stochastic Modeling and the Theory of t M S ( t ) = Queues” Chapter 2 52-130 µ D. R. Cox: “Renewal Processes” Prob { γ t ≤ x } = G s ( x ) Søren Asmussen: “Applied Probability and Queues” Chapter V pp.138-168 Darryl Daley and Vere-Jones: “An Introduction to the Theory of Point Processes” Chapter 4 pp. 66-110 Bo Friis Nielsen Renewal Processes Bo Friis Nielsen Renewal Processes Discrete Renewal Theory (7.6) Discrete Renewal Theory (7.6) (cont.) P { X = k } = p k Let u n be the renewal density ( p 0 = 0) i.e. the probability of n � having an event at time n M ( n ) = p k [ 1 + M ( n − k )] k = 0 n n � u n = δ n + p k u n − k � = F ( n ) + p k M ( n − k ) k = 0 k = 0 A renewal equation. In general Lemma n 7.1 Page 381 If { v n } satisifies v n = b n + � n k = 0 p k v n − k and u n � v n = b n + p k v n − k satisfies u n = δ n + � n k = 0 p k u n − k then k = 0 n The solution is unique, which we can see by solving recursively � v n = b n − k u k b 0 v 0 = k = 0 1 − p 0 b 1 + p 1 v 0 � v 1 = 1 − p 0 Bo Friis Nielsen Renewal Processes Bo Friis Nielsen Renewal Processes

The Discrete Renewal Theorem Theorem 7.1 Page 383 Suppose that 0 < p 1 < 1 and that { u n } and { v n } are the solutions to the renewal equations v n = b n + � n k = 0 p k v n − k and u n = δ n + � n k = 0 p k u n − k , respectively. Then 1 (a) lim n →∞ u n = � ∞ k = 0 kp k � ∞ k = 0 b k (b) if � ∞ k = 0 | b k | < ∞ then lim n →∞ v n = � ∞ k = 0 kp k � Bo Friis Nielsen Renewal Processes