Infinite Markov chains: M / G / 1 structure b 0 a 1 b 1 a 0 a 2 a 3 0 1 · · · i − 1 i i + 1 i + 2 • Transition probabilities P = ( p ij ) b 0 b 1 b 2 b 3 · · · a 0 a 1 a 2 a 3 0 a 0 a 1 a 2 P = 0 0 a 0 a 1 . ... . . • Example: Number in system just after departure in M / G / 1 queue i = 0 a i z i and B ( z ) = � ∞ i = 0 b i z i and assume mean step size = A ( 1 ) ( 1 ) − 1 < 0 • Let A ( z ) = � ∞ • Global balance equations p i = p i + 1 a 0 + p i a 1 + · · · + p 1 a i + p 0 b i , i = 0, 1, 2, ... 20
Infinite Markov chains: M / G / 1 structure b 0 a 1 b 1 a 0 a 2 a 3 0 1 · · · i − 1 i i + 1 i + 2 • Solution: Recursion – Global balance for { 0, 1, ... , i − 1 } p 0 ¯ b i + p 1 ¯ a i − 1 + · · · + p i − 1 ¯ a 1 = p i a 0 , i = 1, 2, ... where ∞ ∞ � � ¯ b i = b i , a i = a i ¯ j = i j = i – Recursively calculate p 1 , p 2 , ... starting with p 0 – p 0 follows from (mean displacement is 0 in equilibrium) p 0 B ( 1 ) ( 1 ) + ( 1 − p 0 )( A ( 1 ) ( 1 ) − 1 ) = 0 21
Infinite Markov chains: M / G / 1 structure b 0 a 1 b 1 a 0 a 2 a 3 0 1 · · · i − 1 i i + 1 i + 2 • Solution: Generating function – Let P ( z ) = � ∞ i = 0 p i z i , A ( z ) = � ∞ i = 0 a i z i , B ( z ) = � ∞ i = 0 b i z i – Multiply balance equations by z i and add all equations P ( z ) = z − 1 ( P ( z ) − p 0 ) A ( z ) + p 0 B ( z ) – Solving this equation P ( z ) = p 0 ( B ( z ) − z − 1 A ( z )) = p 0 ( zB ( z ) − A ( z )) 1 − z − 1 A ( z ) z − A ( z ) – Normalization P ( 1 ) = 1 so 1 − A ( 1 ) ( 1 ) p 0 = B ( 1 ) ( 1 ) + 1 − A ( 1 ) ( 1 ) 22
Infinite Markov chains: G / M / 1 structure a 1 b i a i a 3 a 2 a 0 0 1 · · · i − 2 i − 1 i i + 1 · · · • Transition probabilities P = ( p ij ) b 0 a 0 0 0 · · · b 1 a 1 a 0 0 ∞ � b 2 a 2 a 1 a 0 P = b i = 1 − ( a 0 + · · · + a i ) = a i , b 3 a 3 a 2 a 1 j = i + 1 . ... . . • Example: Number in system just before arrival in G / M / 1 queue • Global balance equations ∞ � p i = p i − 1 a 0 + p i a 1 + p i + 1 a 2 + · · · = p i − 1 + j a j , i = 1, 2, ... j = 0 23
Infinite Markov chains: G / M / 1 structure a 1 b i a i a 3 a 2 a 0 0 1 · · · i − 2 i − 1 i i + 1 · · · • Solution: Geometric – Ratio p i + 1 / p i is expected time spent in i + 1 before first return to i , given initial state is i – Transition structure implies that ratio p i + 1 / p i does not depend on i p i + 1 = α p i – Hence p i = p 0 α i and substitution in global balance equation yields ∞ � α j a j = A (α) α = j = 0 – α is the unique root on ( 0, 1 ) of α = A (α) provided mean step size 1 − A ( 1 ) ( 1 ) < 0 24
Examples of Quasi-Birth-and-Death processes: Single server with setup • Single server • Poisson arrivals with rate λ wich need exponential service times with rate µ • Server switched off when system is empty • Server switched on when first customer arrives which requires exponential setup time with rate θ 25
Examples of Quasi-Birth-and-Death processes: Single server with setup • Markov chain with states ( i , j ) with i number in system and j status of server: 0 means off, 1 means on • Transition rate diagram λ λ λ 0 , 1 1 , 1 2 , 1 i, 1 µ µ µ θ θ θ · · · · · · λ λ λ 0 , 0 1 , 0 2 , 0 i, 0 • Global balance equations p ( 0, 0 )λ = p ( 1, 1 )µ p ( 0, 1 )λ = 0 p ( i , 0 )(λ + θ) = p ( i − 1, 0 )λ p ( i , 1 )(λ + µ) = p ( i − 1, 1 )λ + p ( i , 0 )θ + p ( i + 1, 1 )µ , i = 1, 2, ... 26
Examples of Quasi-Birth-and-Death processes: Single server with setup • Order states as ( 0, 0 ) , ( 0, 1 ) , ( 1, 0 ) , ( 1, 1 ) , ( 2, 0 ) , ( 2, 1 ) , ... and partition into levels i = { ( i , 0 ) , ( i , 1 ) } • Generator Q of the form B 1 A 0 0 0 ... B 2 A 1 A 0 0 A 2 A 1 A 0 Q = 0 0 0 A 2 A 1 . ... . . where � − λ � � 0 0 � � λ 0 � � − (λ + θ) � � 0 0 � 0 θ B 1 = B 2 = A 0 = A 1 = A 2 = , , , , 0 − λ µ 0 0 λ 0 − (λ + µ) 0 µ • Let p = ( p 0 , p 1 , ... ) and p i = ( p ( i , 0 ) , p ( i , 1 )) p 0 B 1 + p 1 B 2 = 0, p i − 1 A 0 + p i A 1 + p i + 1 A 2 = 0, i = 1, 2, ... 27
Examples of Quasi-Birth-and-Death processes: Single server with setup • Solution: Matrix-geometric – Global balance for level 0, 1, ... , i p ( i + 1, 1 )µ = ( p ( i , 0 ) + p ( i , 1 ))λ or in matrix form p i + 1 A 2 = p i A 3 � 0 λ � with A 3 = 0 λ – Substituting in balance equation 0 = p i − 1 A 0 + p i A 1 + p i + 1 A 2 = p i − 1 A 0 + p i ( A 1 + A 3 ) – Hence p i = − p i − 1 A 0 ( A 1 + A 3 ) − 1 = p i − 1 R with � λ/(λ + θ) λ/µ � R = − A 0 ( A 1 + A 3 ) − 1 = 0 λ/µ 28
Examples of Quasi-Birth-and-Death processes: Single server with setup • Solution: Matrix-geometric – Iterating yields p i = p 0 R i , i = 0, 1, 2, ... – Vector p 0 follows from boundary equation p 0 B 1 + p 1 B 2 = p 0 ( B 1 + RB 2 ) = 0 and normalization ∞ ∞ � � � R i e = p 0 ( I − R ) − 1 e p ( i , j ) = p i e = p 0 1 = i , j i = 0 i = 0 where I is identity matrix and e all-one vector 29
Examples of Quasi-Birth-and-Death processes: Single server with setup • Solution: Matrix-analytic – Let g jk be probability of first passage to level i − 1 in ( i − 1, k ) for initial state ( i , j ) at level i � 0 1 � G = ( g jk ) = 0 1 – Then p i + 1 A 2 = p i A 0 G – Substituting in balance equation 0 = p i − 1 A 0 + p i A 1 + p i + 1 A 2 = p i − 1 A 0 + p i ( A 1 + A 0 G ) – Hence p i = − p i − 1 A 0 ( A 1 + A 0 G ) − 1 = p i − 1 R 30
Examples of Quasi-Birth-and-Death processes: Single server with setup • Solution: Spectral expansion – Seek solutions of balance equations p i − 1 A 0 + p i A 1 + p i + 1 A 2 = 0 of the form p i = y · x i , i = 0, 1, 2, ... , where y = ( y ( 0 ) , y ( 1 )) is non-null vector and | x | < 1 – Substitution and dividing by common powers of x yields A 0 + xA 1 + x 2 A 2 � � y = 0 – Desired values of x are roots inside the unit circle of det ( A 0 + xA 1 + x 2 A 2 ) = (λ − (λ + θ) x )(µ x − λ)( x − 1 ) = 0 31
Examples of Quasi-Birth-and-Death processes: Single server with setup • Solution: Spectral expansion – Roots λ x 2 = λ x 1 = λ + θ , µ with corresponding non-null vectors − θ x 1 y 1 = ( 1, ) , y 2 = ( 0, 1 ) λ − (λ + µ) x 1 + µ x 2 1 – Set p i = c 1 y 1 x i 1 + c 2 y 2 x i 2 where coefficients c 1 and c 2 follow from boundary equation p 0 B 1 + p 1 B 2 = 0 and normalization p ( i , j ) = c 1 y 1 e + c 2 y 2 e � 1 = 1 − x 1 1 − x 2 i , j 32
Examples of Quasi-Birth-and-Death processes: Single server with setup • Solution: Generating function – Let P ( z ) = � ∞ i = 0 p i z i – Multiply balance equations by z i and add all equations zP ( z ) A 0 + ( P ( z ) − p 0 ) A 1 + z − 1 ( P ( z ) − p 0 − p 1 z ) A 2 = 0 – Solving this equation P ( z ) A ( z ) = p 0 ( A 1 + z − 1 A 2 ) + p 1 A 2 where � λ z − (λ + θ) � θ A ( z ) = zA 0 + A 1 + z − 1 A 2 = λ z − (λ + µ) + µ z − 1 0 – Vectors p 0 , p 1 and P ( 1 ) follow from p 0 B 1 + p 1 B 2 = 0 P ( 1 ) A ( 1 ) = p 0 ( A 1 + A 2 ) + p 1 A 2 , P ( 1 ) e = 1 P ( 1 ) A ( 1 ) ( 1 ) e = − p 0 A 2 e 33
Examples of Quasi-Birth-and-Death processes: Shortest queue with jockeying • Two parallel exponential servers with rate µ , each with its own queue • Customers arrive according to Poisson stream with rate λ and join shortest queue on arrival • Jockeying: If difference in queues exceeds 1 , then a customer jumps from longest to shortest queue 34
Examples of Quasi-Birth-and-Death processes: Shortest queue with jockeying • Markov chain with states ( i , j ) with i number in shortest queue and j difference between queues • Transition rate diagram 0 , 1 1 , 1 i, 1 i +1 , 1 2 µ λ λ · · · 2 µ · · · µ λ λ 0 , 0 1 , 0 i, 0 i +1 , 0 35
Examples of Quasi-Birth-and-Death processes: Shortest queue with jockeying • Partition into levels i = { ( i , 0 ) , ( i , 1 ) } for i = 0, 1, ... • Generator Q of the form B 1 A 0 0 0 ... A 2 A 1 A 0 0 A 2 A 1 A 0 Q = 0 0 0 A 2 A 1 . ... . . where � − λ � � 0 0 � � − (λ + 2 µ) � � 0 2 µ � λ λ B 1 = A 0 = A 1 = A 2 = , , , µ − (λ + µ) λ 0 2 µ − (λ + 2 µ) 0 0 • Let p = ( p 0 , p 1 , ... ) and p i = ( p ( i , 0 ) , p ( i , 1 )) p 0 B 1 + p 1 A 2 = 0 p i − 1 A 0 + p i A 1 + p i + 1 A 2 = 0, i = 1, 2, ... 36
Examples of Quasi-Birth-and-Death processes: Single server serving n customer types • Single server serving n types of customers, numbered 1, ... , n • Type i customers arrive according to Poisson stream with rate λ i • Type i customers require exponential service with rate µ i 37
Examples of Quasi-Birth-and-Death processes: Single server serving n customer types • Markov chain with states ( i , j ) where i number waiting in the queue and j is customer type in service • State ( 0, 0 ) is empty state • Transition rate diagram where λ = λ 1 + · · · + λ n and p i = λ i /λ λ λ 0 , n 1 , n i, n µ n p n µ n p n . µ n p 1 µ n p 1 . . λ 0 , 1 1 , 1 · · · i, 1 · · · µ 1 p 1 µ 1 p 1 λ 1 µ 1 0 , 0 38
Examples of Quasi-Birth-and-Death processes: Single server serving n customer types • Partition into level 0 = { ( 0, 0 ) , ... , ( 0, n ) } and levels i = { ( i , 1 ) , ... , ( i , n ) } for i = 1, 2, ... • Generator Q of the form B 1 B 0 0 0 ... B 2 A 1 A 0 0 A 2 A 1 A 0 Q = 0 0 0 A 2 A 1 . ... . . where λ 0 − (λ + µ 1 ) 0 µ 1 p 1 · · · µ 1 p n . . ... ... , , . . A 0 = A 1 = A 2 = . . 0 λ 0 − (λ + µ n ) µ n p 1 · · · µ n p n • Let p = ( p 0 , p 1 , ... ) , p 0 = ( p ( 0, 0 ) , ... , p ( 0, n )) and p i = ( p ( i , 1 ) , ... , p ( i , n )) p 0 B 1 + p 1 B 2 = 0 p 0 B 0 + p 1 A 1 + p 2 A 2 = 0 p i − 1 A 0 + p i A 1 + p i + 1 A 2 = 0, i = 2, 3, ... 39
Quasi-Birth-and-Death processes • Markov chain with states ( 0, 0 ) , ( 0, 1 ) , ... , ( 0, n ) , ( 1, 0 ) , ( 1, 1 ) , ... , ( 1, n ) , ( 2, 0 ) , ... • Partition into levels i = { ( i , 0 ) , ... , ( i , n ) } for i = 0, 1, ... • Generator Q of the form B 1 A 0 0 0 ... A 2 A 1 A 0 0 A 2 A 1 A 0 0 Q = 0 0 A 2 A 1 . ... . . where B 1 , A 0 , A 1 and A 2 are n + 1 × n + 1 matrices • Let p = ( p 0 , p 1 , ... ) and p i = ( p ( i , 0 ) , p ( i , 1 ) , ... , p ( i , n )) p 0 B 0 + p 1 A 2 = 0 p i − 1 A 0 + p i A 1 + p i + 1 A 2 = 0, i = 1, 2, ... 40
Quasi-Birth-and-Death processes • Solution: Matrix-geometric – Probability vector p is given by p i = p 0 R i , i = 0, 1, 2, ... – Rate matrix R is minimal nonnegative solution of A 0 + RA 1 + R 2 A 2 = 0 – R jk is expected time spent in ( i + 1, k ) before first return to level i , given initial state is ( i , j ) – Vector p 0 follows from boundary equation and normalization p 0 ( B 1 + RA 2 ) = 0 p 0 ( I − R ) − 1 e = 1 – Rate matrix R can be determined by successive substitution from fixed point equation R = − ( A 0 + R 2 A 2 ) A − 1 1 starting with R = 0 41
Quasi-Birth-and-Death processes • Solution: Matrix-analytic – Let g jk be probability of first passage to level i − 1 in ( i − 1, k ) for initial state ( i , j ) at level i – Matrix G = ( g jk ) is is minimal nonnegative solution of A 2 + A 1 G + A 0 G 2 = 0 – Then p i + 1 A 2 = p i A 0 G – Substituting in balance equation 0 = p i − 1 A 0 + p i A 1 + p i + 1 A 2 = p i − 1 A 0 + p i ( A 1 + A 0 G ) – Probability vector p is given by p i = − p i − 1 A 0 ( A 1 + A 0 G ) − 1 = p i − 1 R with R = − A 0 ( A 1 + A 0 G ) − 1 42
Quasi-Birth-and-Death processes • Solution: Matrix-analytic – Vector p 0 follows from boundary equation and normalization p 0 ( B 0 + RA 2 ) = p 0 ( B 0 + A 0 G ) = 0 p 0 ( I − R ) − 1 e = 1 – Note RA 2 = A 0 G – Matrix G can be determined by successive substitution from fixed point equation G = − ( A 2 + A 0 G 2 ) A − 1 1 starting with G = 0 43
Quasi-Birth-and-Death processes • Solution: Spectral expansion – Seek solutions of balance equations p i − 1 A 0 + p i A 1 + p i + 1 A 2 = 0 of the form p i = y · x i , i = 0, 1, 2, ... , where y = ( y ( 0 ) , ... , y ( n )) is non-null vector and | x | < 1 – Substitution and dividing by common powers of x yields A 0 + xA 1 + x 2 A 2 � � y = 0 – Suppose determinant equation det ( A 0 + xA 1 + x 2 A 2 ) = 0 has exactly n + 1 roots x 0 , ... , x n with | x k | < 1 – Let y 0 , ... , y n be the corresponding non-null vectors A 0 + x k A 1 + x 2 � � y k k A 2 = 0, k = 0, ... , n 44
Quasi-Birth-and-Death processes • Solution: Spectral expansion – Set n � c k y k x i p i = k k = 0 – Coefficients c 0 , ... , c k follow from boundary equation n � p 0 B 1 + p 1 A 2 = c k ( y k B 1 + y k A 2 x k ) = 0 k = 0 and normalization n c k y k e � � p ( i , j ) = 1 = 1 − x k i , j k = 0 45
Quasi-Birth-and-Death processes • Solution: Generating function – Let P ( z ) = � ∞ i = 0 p i z i – Multiply balance equations by z i and add all equations zP ( z ) A 0 + ( P ( z ) − p 0 ) A 1 + z − 1 ( P ( z ) − p 0 − p 1 z ) A 2 = 0 – Solving this equation P ( z ) A ( z ) = p 0 ( A 1 + z − 1 A 2 ) + p 1 A 2 = p 0 ( A 1 − B 1 + z − 1 A 2 ) where A ( z ) = zA 0 + A 1 + z − 1 A 2 46
Quasi-Birth-and-Death processes • Solution: Generating function – Suppose determinant equation det ( A ( z )) = 0 has exactly n + 1 roots z 0 = 1 , z 1 , ... , z n with | z k | < 1 for k = 1, ... , n – Let v 0 = e , v 1 , ... , v n be non-null vectors satisfying A ( z k ) v k = 0 – Vector p 0 follows from p 0 ( A 1 − B 1 + z − 1 k A 2 ) v k = 0, k = 1, ... , n P ( 1 ) A ( 1 ) = p 0 ( A 1 − B 1 + A 2 ) , P ( 1 ) e = 1 P ( 1 ) A ( 1 ) ( 1 ) e = − p 0 A 2 e 47
Quasi-Birth-and-Death processes: More structure • A 0 , A 1 and A 2 are upper triangular: A 0 ( i , j ) = 0 if j < i • Implication: R is also upper triangular • Example: � a 0 a 1 � � b 0 b 1 � � c 0 c 1 � A 0 = , A 1 = , A 2 = , 0 a 2 0 b 2 0 c 2 • Then � r 0 r 1 � R = 0 r 2 where a 0 + r 0 b 0 + r 2 0 c 0 = 0 a 2 + r 2 b 2 + r 2 2 c 2 = 0 a 1 + r 0 b 1 + r 1 b 2 + r 2 0 c 1 + ( r 0 r 1 + r 1 r 2 ) c 2 = 0 48
Quasi-Birth-and-Death processes: More structure • Transition rates A 2 from level i to level i − 1 have the form r 0 r 0 p 0 · · · r 0 p n . . . . � p 0 · · · p n � . . A 2 = rp = = . . . r n r n p 0 · · · r n p n • Example: Single server with setup � 0 0 � � 0 � � 0 1 � A 2 = = 0 µ µ • Example: Shortest queue with jockeying � 0 2 µ � � 2 µ � � 0 1 � A 2 = = 0 0 0 • Example: Single server serving n customer types for which µ 1 p 1 · · · µ 1 p n µ 1 . . . . . = . � p 1 · · · p n � A 2 = . . . µ n p 1 · · · µ n p n µ n 49
Quasi-Birth-and-Death processes: More structure • Transition rates A 2 from level i to level i − 1 have the form r 0 r 0 p 0 · · · r 0 p n . . . . � p 0 · · · p n � . . A 2 = rp = = . . . r n r n p 0 · · · r n p n • Interpretation: Rate of jumping from ( i , j ) to level i − 1 is r j , this jump is to ( i − 1, k ) with probability p k • Important: Jump probability does not depend on j • This implies p 0 · · · p n . . . . = ep G = . . p 0 · · · p n and R = − A 0 ( A 1 + A 0 G ) − 1 = − A 0 ( A 1 + A 0 ep ) − 1 50
Quasi-Birth-and-Death processes: More structure • Transition rates A 0 from level i to level i + 1 have the form r 0 r 0 p 0 · · · r 0 p n . . . . � p 0 · · · p n � . . A 0 = rp = = . . . r n r n p 0 · · · r n p n • Example: Shortest queue with jockeying � 0 0 � � 0 � � 1 0 � A 0 = = λ 0 λ 51
Quasi-Birth-and-Death processes: More structure • Transition rates A 0 from level i to level i + 1 have the form r 0 r 0 p 0 · · · r 0 p n . . . . � p 0 · · · p n � . . A 0 = rp = = . . . r n r n p 0 · · · r n p n • Interpretation: Rate of jumping from ( i , j ) to level i + 1 is r j , this jump is to ( i + 1, k ) with probability p k • This implies r 0 r 0 q 0 · · · r 0 q n . . . . � q 0 · · · q n � . . R = rq = = . . . r n r n q 0 · · · r n q n for some vector q • Hence R i = ( qr ) i − 1 R , p i = p 0 R i = ( qr ) i − 1 p 1 , i = 1, 2, ... • Number x = qr is spectral radius of R which is unique root on ( 0, 1 ) of det ( A 0 + xA 1 + x 2 A 2 ) = 0 52
Quasi-Birth-and-Death processes: Linear rates • Transition rates have linear form to neighboring phases – Rate from ( i , j ) to ( i + k , j + 1 ) is a k ( n − j ) – Rate from ( i , j ) to ( i + k , j ) is b k ( n − j ) + c k j – Rate from ( i , j ) to ( i + k , j − 1 ) with rate d k j • So A 0 is of tri-diagonal form b 1 n a 1 n 0 d 1 b 1 ( n − 1 ) + c 1 a 1 ( n − 1 ) d 1 2 b 1 ( n − 2 ) + c 1 2 0 ... ... ... A 0 = b 1 2 + c 1 ( n − 2 ) a 1 2 0 d 1 ( n − 1 ) b 1 + c 1 ( n − 1 ) a 1 0 d 1 n c 1 n 53
Quasi-Birth-and-Death processes: Linear rates • Transition rate diagram i, j + 1 a − 1 ( n − j ) a 1 ( n − j ) i, j i + 1 , j b 1 j + c 1 ( n − j ) d 0 j i, j − 1 54
Quasi-Birth-and-Death processes: Example with linear rates • n parallel servers serving two customer types 1 and 2 • Type 1 customers require exponential service with rate µ 1 , type 2 with rate µ 2 • Customers arrive as Poisson stream with rate λ , fraction p 1 is of type 1 and p 2 of type 2 • States ( i , j ) with i number of customers in the queue and j number of type 1 customers in service • Transition rate diagram i, j + 1 ( n − j ) µ 2 p 1 i, j i − 1 , j i + 1 , j jµ 1 p 1 + ( n − j ) µ 2 p 2 λ jµ 1 p 2 i, j − 1 • Non-zero coefficients a − 1 = µ 2 p 1 , b 1 = c 1 = λ/ n , b − 1 = µ 2 p 2 , c − 1 = µ 1 p 1 , d − 1 = µ 1 p 2 55
Quasi-Birth-and-Death processes: Example with linear rates • n parallel servers and Poisson arrivals with rate λ which need exponential service times with rate µ • Server switched off when queue is empty • Idle server switched on when customer arrives which requires exponential setup time with rate θ • States ( i , j ) with i number of customers in the queue and j number of active servers • Transition rate diagram i, j + 1 ( n − j ) θ i, j i − 1 , j i + 1 , j jµ λ i, j − 1 • Non-zero coefficients a 0 = θ , b 1 = c 1 = λ/ n , c − 1 = µ 56
Quasi-Birth-and-Death processes: Example with linear rates • Solution: Spectral expansion – Seek solutions of balance equations of the form p i = y · x i , i = 0, 1, 2, ... , where y = ( y ( 0 ) , ... , y ( n )) is non-null vector and | x | < 1 – Substitution and dividing by common powers of x yields following equations for j = 0, · · · , n 0 = ( j + 1 ) D ( x ) y ( j + 1 ) + (( n − j ) B ( x ) + jC ( x )) y ( j ) − (( n − j )( A ( 1 ) + B ( 1 )) + j ( C ( 1 ) + D ( 1 ))) xy ( j ) + ( n − j + 1 ) A ( x ) y ( j − 1 ) where A ( x ) = a 1 + a 0 x + a − 1 x 2 , B ( x ) = b 1 + b 0 x + b − 1 x 2 , C ( x ) = c 1 + c 0 x + c − 1 x 2 , D ( x ) = d 1 + d 0 x + d − 1 x 2 – Solve equations for y by generating function n � y ( j ) z j Y ( z ) = i = 0 57
Quasi-Birth-and-Death processes: Example with linear rates • Solution: Spectral expansion – Multiply equations by z j and add all equations D ( x ) Y ′ ( z ) + nB ( x ) Y ( z ) + ( C ( x ) − B ( x )) zY ′ ( z ) − ( A ( 1 ) + B ( 1 )) nxY ( z ) + ( A ( 1 ) + B ( 1 ) − C ( 1 ) − D ( 1 )) xzY ′ ( z ) + nA ( x ) zY ( z ) − A ( x ) z 2 Y ′ ( z ) = 0 – This equation can be rewritten as Y ′ ( z ) n [ A ( x ) z + B ( x ) − ( A ( 1 ) + B ( 1 )) x ] = A ( x ) z 2 − (( A ( 1 ) + B ( 1 ) − C ( 1 ) − D ( 1 )) x + C ( x ) − B ( x )) z − D ( x ) Y ( z ) z − z 1 ( x ) + n − E ( x ) E ( x ) = z − z 2 ( x ) where z 1 ( x ) and z 2 ( x ) are the roots of the denominator and E ( x ) satisfies 2 E ( x ) − n = n · B ( x ) + C ( x ) − x ( A ( 1 ) + B ( 1 ) + C ( 1 ) + D ( 1 )) , � F ( x ) 2 + 4 A ( x ) D ( x ) where F ( x ) = ( A ( 1 ) + B ( 1 ) − C ( 1 ) − D ( 1 )) x + C ( x ) − B ( x ) 58
Quasi-Birth-and-Death processes: Example with linear rates • Solution: Spectral expansion – Differential equation Y ′ ( z ) z − z 1 ( x ) + n − E ( x ) E ( x ) = Y ( z ) z − z 2 ( x ) – General solution Y ( z ) = C ( z − z 1 ( x )) E ( x ) ( z − z 2 ( x )) n − E ( x ) – Key observation: Y ( z ) is a polynomial in z , so exponent E ( x ) = k for k = 0, ... , n – For each k = 0, ... , n the equation 2 k − n = n · B ( x ) + C ( x ) − x ( A ( 1 ) + B ( 1 ) + C ( 1 ) + D ( 1 )) , � F ( x ) 2 + 4 A ( x ) D ( x ) has a unique solution x = x k in the interval ( 0, 1 ) 59
Quasi-Birth-and-Death processes: Example with linear rates • Solution: Spectral expansion – For each k = 0, ... , n the equation 2 k − n = n · B ( x ) + C ( x ) − x ( A ( 1 ) + B ( 1 ) + C ( 1 ) + D ( 1 )) , � F ( x ) 2 + 4 A ( x ) D ( x ) has a unique solution x = x k in the interval ( 0, 1 ) 6 4 2 x 2 x 1 x 0 0 1 x 4 x 3 − 2 − 4 − 6 60
Two-dimensional Markov chain: Re-entrant line (1) µ 2 ∞ µ 1 µ 3 • Two stations, station 1 with two servers, labeled 1 and 3 and station 2 with single server, labeled server 2 • Queue i is served at exponential rate µ i • Queue 1 has infinite supply and customers subsequently go through queue 1, 2 and 3 61
Two-dimensional Markov chain: Re-entrant line (1) j µ 2 µ 1 µ 1 µ 3 µ 3 µ 2 µ 1 i • Markov chain with states ( i , j ) where i number in queue 2 and j number in queue 3 • Global balance equations p ( i , j )(µ 1 + µ 2 + µ 3 ) = p ( i − 1, j )µ 1 + p ( i , j + 1 )µ 3 + p ( i + 1, j − 1 )µ 2 , i > 0, j > 0 p ( i , 0 )(µ 1 + µ 2 ) = p ( i − 1, 0 )µ 1 + p ( i , 1 )µ 3 , i > 0 p ( 0, j )(µ 1 + µ 3 ) = p ( 0, j + 1 )µ 3 + p ( 1, j − 1 )µ 2 , j > 0 p ( 0, 0 )µ 1 = p 0,1 µ 3 62
Two-dimensional Markov chain: Re-entrant line (1) j µ 2 µ 1 µ 1 µ 3 µ 3 µ 2 µ 1 i • Substitution of trial solution p ( i , j ) = α i β j in global balance equations αβ(µ 1 + µ 2 + µ 3 ) = βµ 1 + αβ 2 µ 3 + α 2 µ 2 α(µ 1 + µ 2 ) = µ 1 + αβµ 3 β(µ 1 + µ 3 ) = β 2 µ 3 + αµ 2 • Equations solved by α = µ 1 µ 2 and β = µ 1 µ 3 • Normalization yields p ( i , j ) = ( 1 − α)α i ( 1 − β)β j 63
Two-dimensional Markov chain: Re-entrant line (1) • Global balance equations p ( i , j )(µ 1 + µ 2 + µ 3 ) = p ( i − 1, j )µ 1 + p ( i , j + 1 )µ 3 + p ( i + 1, j − 1 )µ 2 , i > 0, j > 0 • Partial balance – Rate out of ( i , j ) due to arrival in queue 2 is equal to rate into ( i , j ) due to departure from queue 3 p ( i , j )µ 1 = p ( i + 1, j )µ 3 – Rate out of ( i , j ) due to departure from queue 2 is equal to rate into ( i , j ) due to arrival in queue 2 p ( i , j )µ 2 = p ( i − 1, j )µ 1 – Rate out of ( i , j ) due to departure from queue 3 is equal to rate into ( i , j ) due to arrival in queue p ( i , j )µ 3 = p ( i + 1, j − 1 )µ 2 • Partial balance yields candidate solution � j � j � µ 1 � i + j � i � µ 1 � j � µ 2 � µ 2 � µ 1 p ( i , j ) = p ( i + j , 0 ) = p ( 0, 0 ) = p ( 0, 0 ) µ 3 µ 3 µ 2 µ 2 µ 3 64
Two-dimensional Markov chain: Re-entrant line (2) µ 1 µ 2 ∞ µ 3 • Two stations, both stations with single server • Queue i is served at exponential rate µ i • Queue 1 has infinite supply and customers subsequently go through queue 1, 2 and 3 • Customers in queue 3 have preemptive priority over customers in queue 1 (Last Buffer First Served) • Assumption 1 µ 1 + 1 µ 3 > 1 µ 2 65
Two-dimensional Markov chain: Re-entrant line (2) j µ 2 µ 3 µ 3 µ 2 µ 1 i • Markov chain with states ( i , j ) where i number in queue 2 and j number in queue 3 • Global balance equations p ( i , j )(µ 2 + µ 3 ) = p ( i , j + 1 )µ 3 + p ( i + 1, j − 1 )µ 2 , i > 0, j > 0 p ( i , 0 )(µ 1 + µ 2 ) = p ( i − 1, 0 )µ 1 + p ( i , 1 )µ 3 , i > 0 p ( 0, j )µ 3 = p ( 0, j + 1 )µ 3 + p ( 1, j − 1 )µ 2 , j > 0 p ( 0, 0 )µ 1 = p ( 0, 1 )µ 3 66
Two-dimensional Markov chain: Re-entrant line (2) j µ 2 µ 3 µ 3 µ 2 µ 1 i • Substitution of trial solution p ( i , j ) = α i β j in global balance equations for i > 0 β(µ 2 + µ 3 ) = β 2 µ 3 + αµ 2 α(µ 1 + µ 2 ) = µ 1 + αβµ 3 • Equations solved by α = µ 1 µ 2 ( 1 − β) and β is the root on ( 0, 1 ) of β 2 µ 3 − β(µ 1 + µ 2 + µ 3 ) + µ 1 = 0 � (µ 1 + µ 2 + µ 3 ) 2 − 4 µ 1 µ 3 β = µ 1 + µ 2 + µ 3 − 2 µ 3 67
Two-dimensional Markov chain: Re-entrant line (2) j µ 2 µ 3 µ 3 µ 2 µ 1 i • Substitution of p ( i , j ) = α i β j for i > 0 in global balance equations for i = 0 p ( 0, j )µ 3 = p ( 0, j + 1 )µ 3 + αβ j − 1 µ 2 , j > 0 • Equations are solved by p ( 0, j ) = c β j − 1 where c = µ 1 µ 3 • Normalization yields µ 3 µ 1 + µ 3 ( 1 − α) , i = 0, j = 0 µ 1 µ 1 + µ 3 ( 1 − α)β j − 1 , i = 0, j > 0 p ( i , j ) = µ 3 µ 1 + µ 3 ( 1 − α)α i β j , i > 0, j ≥ 0 68
Two-dimensional Markov chain: Single server priority queue • Single server serving type 1 and type 2 customers • Type i customers arrive according to Poisson stream with rate λ i • Type i customers require exponential service with rate µ i • Type 1 customers have preemptive priority over type 2 customers 69
Two-dimensional Markov chain: Single server priority queue j λ 2 λ 2 λ 1 λ 1 µ 1 µ 2 λ 2 λ 1 i µ 1 • Markov chain with states ( i , j ) where i number of type 1 and j number of type 2 • Global balance equations p ( i , j )(λ 1 + λ 2 + µ 1 ) = p ( i − 1, j )λ 1 + p ( i + 1, j )µ 1 + p ( i , j − 1 )λ 2 , i > 0, j > 0 p ( i , 0 )(λ 1 + λ 2 + µ 1 ) = p ( i − 1, 0 )λ 1 + p ( i + 1, 0 )µ 1 , i > 0 p ( 0, j )(λ 1 + λ 2 + µ 2 ) = p ( 1, j )µ 1 + p ( 0, j − 1 )λ 2 + p ( 0, j + 1 )µ 2 , j > 0 p ( 0, 0 )(λ 1 + λ 2 ) = p ( 0, 1 )µ 2 + p ( 1, 0 )µ 1 • Note that p ( 0, 0 ) = 1 − ρ 1 − ρ 2 where ρ i = λ i 70 µ i
Two-dimensional Markov chain: Single server priority queue • Solution: Difference equation – Balance equations for j = 0 are homogeneous second-order difference equation p ( i , 0 )(λ 1 + λ 2 + µ 1 ) = p ( i − 1, 0 )λ 1 + p ( i + 1, 0 )µ 1 , i > 0 – General solution p ( i , 0 ) = c 0,0 x i 0 + c 0,1 x i 1 , i ≥ 0 where 0 < x 0 < 1 < x 1 are roots of quadratic equation x (λ 1 + λ 2 + µ 1 ) = λ 1 + x 2 µ 1 – Convergence of p ( i , 0 ) implies c 0,1 = 0 so p ( i , 0 ) = c 0,0 x i 0 = ( 1 − ρ 1 − ρ 2 ) x i 0 , i ≥ 0 71
Two-dimensional Markov chain: Single server priority queue • Solution: Difference equation – Balance equations for j = 1 are inhomogeneous second-order difference equation p ( i , 1 )(λ 1 + λ 2 + µ 1 ) = p ( i − 1, 1 )λ 1 + p ( i + 1, 1 )µ 1 + p ( i , 0 )λ 2 = p ( i − 1, 1 )λ 1 + p ( i + 1, 1 )µ 1 + c 0,0 x i 0 λ 2 , i > 0 – General solution p ( i , 1 ) = c 1,0 x i 0 + c 1,1 ( i + 1 ) x i 0 � i + 1 � � i + 1 � x i x i = c 1,0 0 + c 1,1 i ≥ 0 0 , 0 1 where λ 2 c 0,0 c 1,1 = λ 1 + λ 2 − 2 µ 1 x 0 – Coefficient c 1,0 follows from balance equation in ( 0, 0 ) c 1,0 = λ 1 + λ 2 − µ 1 x 0 c 0,0 − c 1,1 µ 2 72
Two-dimensional Markov chain: Single server priority queue • Solution: Difference equation – Repeating procedure for balance equations for j = 2, 3, ... leads to j � i + j � � x i p ( i , j ) = c j , k 0 , i , j ≥ 0 k k = 0 where coefficients c j , k can be determined recursively starting with c 0,0 = 1 − ρ 1 − ρ 2 73
Two-dimensional Markov chain: Single server priority queue • Solution: Generating function j = 0 p ( i , j ) x i y j for | x | , | y | ≤ 1 – Let P ( x , y ) = � ∞ � ∞ i = 0 – Multiply balance equations by x i y j and add all equations h 1 ( x , y ) P ( x , y ) = h 2 ( x , y ) P ( 0, y ) + h 3 ( x , y ) P ( 0, 0 ) with h 1 ( x , y ) = λ 1 xy ( 1 − x ) + λ 2 xy ( 1 − y ) − µ 1 y ( 1 − x ) h 2 ( x , y ) = − µ 1 y ( 1 − x ) + µ 2 x ( 1 − y ) h 3 ( x , y ) = − µ 2 x ( 1 − y ) and P ( 0, 0 ) = p ( 0, 0 ) = 1 − ρ 1 − ρ 2 74
Two-dimensional Markov chain: Single server priority queue • Solution: Generating function – For every | y | ≤ 1 let x = ξ( y ) be the unique root with | x | ≤ 1 of the quadratic equation 0 = h 1 ( x , y ) = λ 1 xy ( 1 − x ) + λ 2 xy ( 1 − y ) − µ 1 y ( 1 − x ) – Then � (λ 1 + µ 1 + λ 2 ( 1 − y )) 2 − 4 λ 1 µ 1 x = ξ( y ) = λ 1 + µ 1 + λ 2 ( 1 − y ) − 2 λ 1 – Substituting x = ξ( y ) and P ( 0, 0 ) = 1 − ρ 1 − ρ 2 in h 1 ( x , y ) P ( x , y ) = h 2 ( x , y ) P ( 0, y ) + h 3 ( x , y ) P ( 0, 0 ) gives P ( 0, y ) = − h 3 (ξ( y ) , y )( 1 − ρ 1 − ρ 2 ) h 2 (ξ( y ) , y ) 75
Two-dimensional Markov chain: Single server priority queue • Solution: Matrix geometric – Partition states space into levels i = { ( i , 0 ) , ( i , 1 ) , ... } – Generator Q of the form B 1 A 0 0 0 ... A 2 A 1 A 0 0 A 2 A 1 A 0 0 Q = 0 0 A 2 A 1 . ... . . where ( λ = λ 1 + λ 2 ) λ 1 − λ − µ 1 λ 2 µ 1 , , λ 1 ... − λ − µ 1 λ 2 µ 1 ... A 0 = A 1 = A 2 = ... ... − λ λ 2 µ 2 − λ − µ 2 λ 2 B 1 = ... ... ... 76
Two-dimensional Markov chain: Single server priority queue • Solution: Matrix geometric – A 0 , A 1 and A 2 are upper triangular and have repeating structure – Implication: R is also upper triangular and of the form r 0 r 1 r 2 ... r 0 r 1 ... R = r 0 ... ... – Elements can be determined recursively from matrix equation A 0 + RA 1 + R 2 A 2 = 0 µ 1 r 2 0 − (λ + µ 1 ) r 0 + λ 1 = 0 k � µ 1 r k − l r l − (λ + µ 1 ) r k + λ 2 r k − 1 = 0, k ≥ 1 l = 0 77
Two-dimensional Markov chain: Single server priority queue • Solution: Matrix geometric – Vector p 0 = ( p ( 0, 0 ) , p ( 0, 1 ) , ... ) follows from boundary equations 0 = p 0 B 1 + p 1 A 2 = p 0 ( B 1 + RA 2 ) yielding the recursion p ( 0, 1 )µ 2 = p ( 0, 0 )(λ − µ 1 r 0 ) j � p ( 0, j + 1 )µ 2 = (λ + µ 2 ) p ( 0, j ) − λ 2 p ( 0, j − 1 ) − µ 1 p ( 0, j − k ) r k , j ≥ 1 k = 0 starting with p ( 0, 0 ) = 1 − ρ 1 − ρ 2 – Vector p i + 1 = ( p ( 0, 0 ) , p ( 0, 1 ) , ... ) can be obtained from p i + 1 = p i R yielding j � p ( i + 1, j ) = p ( i , j − k ) r k k = 0 78
Two-dimensional Markov chain: Single server priority queue • Solution: Matrix analytic – A 0 , A 1 and A 2 are upper triangular and have repeating structure – Implication: G is also upper triangular and of the form g 0 g 1 g 2 ... g 0 g 1 ... G = g 0 ... ... – g k is probability of first passage to level i − 1 in ( i − 1, j + k ) for initial state ( i , j ) at level i – One-step analysis µ 1 λ 1 g 2 g 0 = + 0 λ + µ 1 λ + µ 1 k λ 2 λ 1 � g k = g k − 1 + g l g k − l , k ≥ 1 λ + µ 1 λ + µ 1 l = 0 79
Two-dimensional Markov chain: Single server priority queue • Solution: Matrix analytic – From p i + 1 A 2 = p i A 0 G follows j � p ( i + 1, j )µ 1 = p ( i , j − k )λ 1 g k , i , j , ≥ 0 k = 0 80
Two-dimensional Markov chain: Single server priority queue • Solution: Matrix analytic – Vector p 0 follows from boundary equations 0 = p 0 B 1 + p 1 A 2 = p 0 ( B 1 + A 0 G ) – B 1 + A 0 G is generator of Markov chain embedded on level 0 (0 , j + k ) λ 1 g k λ 2 (0 , j ) µ 2 – Global balance for { ( 0, 0 ) , ( 0, 1 ) , ... , ( 0, j ) } j � k � � � p ( 0, j + 1 )µ 2 = p ( 0, j )λ 2 + p ( 0, j − k )λ 1 1 − g l k = 0 l = 0 81
Two-dimensional Markov chain: Shortest queue µ λ µ • Two parallel exponential servers with rate µ , each with its own queue • Customers arrive according to Poisson stream with rate λ and join shortest queue on arrival 82
Two-dimensional Markov chain: Shortest queue µ µ λ λ µ 2 µ λ • Markov chain with states ( i , j ) where i number in shortest queue and j difference between queues • Global balance equations p ( i , j )(λ + 2 µ) = p ( i − 1, j + 1 )λ + p ( i , j + 1 )µ + p ( i + 1, j − 1 )µ , i > 0, j > 1 p ( i , 1 )(λ + 2 µ) = p ( i − 1, 2 )λ + p ( i , 2 )µ + p ( i , 0 )λ + p ( i + 1, 0 ) 2 µ , i > 0 p ( i , 0 )(λ + 2 µ) = p ( i − 1, 1 )λ + p ( i , 1 )µ , i > 0 p ( 0, j )(λ + µ) = p ( 0, j + 1 )µ + p ( 1, j − 1 )µ , j > 1 83
Two-dimensional Markov chain: Shortest queue µ µ λ λ µ 2 µ λ • Markov chain with states ( i , j ) where i number in shortest queue and j difference between queues • Global balance equations p ( i , j )(λ + 2 µ) = p ( i − 1, j + 1 )λ + p ( i , j + 1 )µ + p ( i + 1, j − 1 )µ , i > 0, j > 1 p ( i , 1 )(λ + 2 µ) = p ( i − 1, 2 )λ + p ( i , 2 )µ λ 2 µ + ( p ( i − 1, 1 )λ + p ( i , 1 )µ) λ + 2 µ + ( p ( i , 1 )λ + p ( i + 1, 1 )µ) λ + 2 µ , i > 0 p ( 0, j )(λ + µ) = p ( 0, j + 1 )µ + p ( 1, j − 1 )µ , j > 1 84
Intermezzo: Method of images φ=0 ∆φ=0 p q • What is the potential Φ in point p ? 85
Intermezzo: Method of images φ=0 ∆φ=0 p −q q • What is the potential Φ in point p ? • Add image charge − q : • Potential Φ in point p is Φ ( p ) = q − q r 1 r 2 where r 1 and r 2 are distances from point p to charge q and − q 86
Intermezzo: Method of images φ=0 ∆φ=0 p −q q q −q • For two planes at right angles Φ ( p ) = q − q + q − q r 1 r 2 r 3 r 4 87
Intermezzo: Method of images A B a b c Φ=Φ Φ=0 a • What is the potential Φ in any point p outside the spheres A and B ? 88
Intermezzo: Method of images A B a b c α 0 Φ=Φ Φ=0 a • What is the potential Φ in any point p outside the spheres A and B ? • Add image charge α 0 in center of A with α 0 = a Φ a : Then Φ ( p ) ≈ α 0 r 0 where r 0 is distance from p to charge α 0 (correct if c = ∞ ) • Potential of α 0 does not vanish on B 89
Intermezzo: Method of images A B a b c α0 β 0 Φ=Φ Φ=0 a • Place new image charge β 0 in B to compensate for this error Φ ( p ) ≈ α 0 + β 0 r 0 r 1 where r 0 and r 1 are distances from p to charges α 0 and β 0 • Charge β 0 alters potential on A 90
Intermezzo: Method of images A B a b c α0 α β 1 0 Φ=Φ Φ=0 a • Place new image charge β 0 in B to compensate for this error Φ ( p ) ≈ α 0 + β 0 r 0 r 1 where r 0 and r 1 are distances from p to charges α 0 and β 0 • Charge β 0 alters potential on A • Place new image charge α 1 in A to compensate for this error Φ ( p ) ≈ α 0 + β 0 + α 1 r 0 r 1 r 2 91
Intermezzo: Method of images A B a b c α0 α β 1 0 Φ=Φ Φ=0 a • Potential problem of two spheres is solved by infinite sequence of image charges in A and B Φ ( p ) = α 0 + β 0 + α 1 + β 1 + · · · r 0 r 1 r 2 r 3 92
Two-dimensional Markov chain: Shortest queue µ µ λ λ µ 2 µ λ • Image charges are products α i β j satisfying the global balance equations p ( i , j )(λ + 2 µ) = p ( i − 1, j + 1 )λ + p ( i , j + 1 )µ + p ( i + 1, j − 1 )µ , i > 0, j > 1 • Substituting α i β j in this equation and dividing by common powers gives the curve αβ(λ + 2 µ) = β 2 λ + αβ 2 µ + α 2 µ 93
Two-dimensional Markov chain: Shortest queue • Quadratic curve αβ(λ + 2 µ) = β 2 λ + αβ 2 µ + α 2 µ 94
Two-dimensional Markov chain: Shortest queue • Unique product form satisfying global balance equations for i > 0, j > 1 and i > 0, j = 1 : 0 β j p ( i , j ) ≈ c 0 α i 0 where α 0 = ρ 2 , β 0 = ρ 2 /( 2 + ρ) with ρ = λ/ 2 µ 0 β j • Product form c 0 α i 0 violates global balance equations for i = 0 95
Two-dimensional Markov chain: Shortest queue 1 β j • Add new product form c 1 α i 1 such that sum satisfies balance equations for i = 0 0 β j 1 β j p ( i , j ) ≈ c 0 α i 0 + c 1 α i 1 • Then β 1 = β 0 and α 1 is companion root of α 0 and c 1 = − α 1 − β 0 c 0 α 0 − β 0 1 β j • Product form c 1 α i 1 violates global balance equations for j = 1 96
Two-dimensional Markov chain: Shortest queue 2 β j • Add new product form c 2 α i 2 such that sum satisfies balance equations for j = 1 0 β j 1 β j 2 β j p ( i , j ) ≈ c 0 α i 0 + c 1 α i 1 + c 2 α i 2 • Then α 2 = α 1 and β 2 is companion root of β 1 and (with ρ = λ/( 2 µ) ) c 2 = − (ρ + α 1 )/β 2 − ( 1 + ρ) (ρ + α 1 )/β 1 − ( 1 + ρ) c 1 2 β j • Product form c 2 α i 2 violates global balance equations for i = 0 97
Two-dimensional Markov chain: Shortest queue • Iterating yields infinite sequence 0 β j 1 β j 2 β j 3 β j p ( i , j ) = c 0 α i 0 + c 1 α i 1 + c 2 α i 2 + c 3 α i 3 + · · · 98
Two-dimensional Markov chain: Random walk in quarter plane q i,j • Equilibrium probabilities can be expressed as infinite sequence of product forms 0 β j 1 β j 2 β j p ( i , j ) = c 0 α i 0 + c 1 α i 1 + c 2 α i 2 + · · · 99
Two-dimensional Markov chain: Random walk in quarter plane q i,j • Equilibrium probabilities can be expressed as infinite sequence of product forms 0 β j 1 β j 2 β j p ( i , j ) = c 0 α i 0 + c 1 α i 1 + c 2 α i 2 + · · · • Provided there are no transitions to North, North-East and East q 0,1 = q 1,1 = q 1,0 = 0 100
Recommend
More recommend
