Markov process · · · In the definition of a Markov process we stated that the next state only depends on the current state, and not on how long we have been already in that state. This means that in a Markov process, the state residence times ( sojourn times ) must be random variables that have a memoryless distribution. We will show that the residence times in continuous-time Markov chain need to be exponentially distributed and in a discrete-time Markov chain need to be geometrically distributed. An extension of Markov processes can be imagined in which the state residence time distributions are not exponential or geometric any more. ◮ In that case it is important to know how long we have been in a particular state and we speak of semi-Markov processes . R.B. Lenin (rblenin@daiict.ac.in) () Queueing Models Part - 1 Autumn 2007 13 / 49
Renewal processes Definition (Renewal processes) A renewal process is a discrete-time stochastic process { X n , n = 1 , 2 , . . . } , where X 1 , X 2 , . . . are independent, identically distributed, nonnegative random variables. Assume that all the random variables X i are distributed as the random variable X with underlying distribution function F X ( x ). We shall interpret X n as the time between ( n − 1) st and n th renewal (event). R.B. Lenin (rblenin@daiict.ac.in) () Queueing Models Part - 1 Autumn 2007 14 / 49
Renewal processes · · · Definition (Coefficient of variation) The coefficient of variation C v of a random variable X is defined as the ratio of the standard deviation σ X to the mean µ X . C v = σ X . µ X A renewal process can be split into a number of less intensive renewal processes. ◮ Let α i ∈ (0 , 1] and � n i =1 α i = 1 (pmf) and C 2 v be the squared coefficient of variation of the renewal time distribution. If we have renewal process with rate λ , we can split it into n ∈ ◆ + renewal processes with rate α i λ and squared coefficient of variation α i C 2 + (1 − α i ), respectively. R.B. Lenin (rblenin@daiict.ac.in) () Queueing Models Part - 1 Autumn 2007 15 / 49
Renewal processes · · · Example (Poisson process) Whenever the times between renewals are exponentially distributed, the renewal process is called a Poisson process . F X ( t ) = 1 − e − λ t , t ≥ 0 and f X ( t ) = λ e − λ t . f ( n ) X ( t ) is the pdf S n = X 1 + · · · + X n which is nothing but an Erlang- n distribution. λ n ∴ f ( n ) ( n − 1)! t n − 1 e − λ t . X ( t ) = R.B. Lenin (rblenin@daiict.ac.in) () Queueing Models Part - 1 Autumn 2007 16 / 49
Renewal processes · · · Example (Poisson process · · · ) It can be proved that F ( n ) X ( t ) = Pr { X 1 + X 2 + · · · + X n ≤ t } � t f ( n ) = X ( u ) du 0 n − 1 ( λ t ) i � e − λ t = . i ! i =0 ∴ Pr { X ( t ) = n } = Pr { X 1 + · · · + X n ≤ t < X 1 + · · · + X n +1 } = Pr { X 1 + · · · + X n +1 ≤ t } − Pr { X 1 + · · · + X n ≤ t } F ( n +1) ( t ) − F ( n ) = X ( t ) X ( λ t ) n e − λ t . = n ! R.B. Lenin (rblenin@daiict.ac.in) () Queueing Models Part - 1 Autumn 2007 17 / 49
Recommend
More recommend
