Mean-Payoff Optimization in Continuous-Time Markov Chains with - PDF document

Mean-Payoff Optimization in Continuous-Time Markov Chains with Parametric Alarms Christel Baier Clemens Dubslaff L ’uboˇ s Korenˇ ciak ech ˇ Anton´ ın Kuˇ cera Vojtˇ Reh´ ak IFIP WG 2.2 2017 Meeting, Bordeaux Anton´ ın Kuˇ cera Mean Payoff in CTMC with Alarms IFIP WG 2.2 2017 1 / 10 Dynamic power management of a disk drive 4 4 4 4 4 new new new new new A N new , 6 A 0 A 1 A 2 A 3 done done done done done wake , 4 wake , 4 wake , 4 wake , 4 sleep new , 6 S 0 S 1 S 2 S 3 S N new new new new new 2 2 2 2 2 What timeout values achieve the minimal long-run average power consumption? Anton´ ın Kuˇ cera Mean Payoff in CTMC with Alarms IFIP WG 2.2 2017 2 / 10

CTMC with parametric alarms (1) “Ordinary” CTMC CTMC with alarms { a 1 , . . . , a n } λ λ s 1 s 1 0 . 4 λ 0 . 4 s 0 λ λ 0 . 6 0 . 6 s 2 s 2 λ s 0 λ a s 3 0 . 7 λ 0 . 3 s 4 Anton´ ın Kuˇ cera Mean Payoff in CTMC with Alarms IFIP WG 2.2 2017 3 / 10 CTMC with parametric alarms (2) In CTMC with parametric alarms { a 1 , . . . , a n } , the distributions associated to { a 1 , . . . , a n } are not fixed but parameterized by a single parameter. Restrictions: At most one alarm is active in each state. Each alarm is set in precisely one state. After fixing the parameters, we obtain a fully stochastic CTMC with alarms. Can we compute parameter values achieving ε -optimal mean-payoff? Anton´ ın Kuˇ cera Mean Payoff in CTMC with Alarms IFIP WG 2.2 2017 4 / 10

Computing ε -optimal parameter values 1. Given a CTMC with parametric alarms and ε > 0, we compute a discretization constant κ > 0 such that ε -optimal parameter values are among the (finitely many) κ -discretized values. 2. We construct a semi-Markov decision process M where the actions correspond to discretized parameter values. Thus, the original problem reduces to computing an optimal strategy for M . 3. The set of states of M is small but the number of actions is very large. We employ a symbolic technique which avoids the explicit construction of these actions. Anton´ ın Kuˇ cera Mean Payoff in CTMC with Alarms IFIP WG 2.2 2017 5 / 10 Computing ε -optimal parameter values (2) 4 4 4 4 4 new new new new new A 0 A 1 A 2 A 3 A N new , 6 done done done done done sleep wake , 4 wake , 4 wake , 4 wake , 4 new , 6 S 0 S 1 S 2 S 3 S N new new new new new 2 2 2 2 2 A 0 A 1 A 2 A 3 A N d s d w Π( d ) , Θ( d ) , e ( d ) S 0 S 1 new Anton´ ın Kuˇ cera Mean Payoff in CTMC with Alarms IFIP WG 2.2 2017 6 / 10

Computing ε -optimal parameter values (3) A 0 A 1 A 2 A 3 A N d s d w S 0 S 1 new An optimal for M can be computed by strategy iteration. Each action d is ranked by a function F ( d ) depending on Π( d ), Θ( d ), and e ( d ). The goal is to find an action with minimal F ( d ). We express F ( d ) analytically, compute its derivative, and consider only a small set of actions close to the local minima of F ( d ). Applicable to alarms with Dirac (fixed-delay), uniform, and Weilbull distributions. Anton´ ın Kuˇ cera Mean Payoff in CTMC with Alarms IFIP WG 2.2 2017 7 / 10 Experiments (disk drive example) We considered N ∈ { 2 , 4 , 6 , 8 } , ε ∈ { 0 . 1 , 0 . 01 , 0 . 001 , 0 . 0005 } . The upper and lower bounds for the timeouts were 0 . 1 and 10 time units, respectively. The required discretization step ranges from 10 − 25 to 10 − 19 . Anton´ ın Kuˇ cera Mean Payoff in CTMC with Alarms IFIP WG 2.2 2017 8 / 10

Experiments (disk drive example), cont. creating solving poly N ε time [s] time [s] degree 0.1 0.15 0.24 46 0.01 0.15 0.25 46 2 0.001 0.16 0.28 53 0.0005 0.16 0.33 53 0.1 0.14 0.25 46 0.01 0.16 0.25 46 4 0.001 0.16 0.28 53 0.0005 0.16 0.33 53 0.1 0.16 0.35 46 0.01 0.16 0.35 46 6 0.001 0.17 0.40 53 0.0005 0.18 0.43 53 0.1 0.19 0.35 46 0.01 0.19 0.35 46 8 0.001 0.20 0.43 53 0.0005 0.22 0.44 53 Anton´ ın Kuˇ cera Mean Payoff in CTMC with Alarms IFIP WG 2.2 2017 9 / 10 Limitations, future work Each alarm has to be set in precisely one state. Hence, we cannot model systems of concurrently running components. POMPD techniques might help? Other objectives? Multi-criteria parameter optimizations. Anton´ ın Kuˇ cera Mean Payoff in CTMC with Alarms IFIP WG 2.2 2017 10 / 10