Motivation Definitions and Problem Statement Existing Algorithms Our Algorithm Empirical Evaluation Conclusion Continuous Optimal Timing Yuliya Butkova, Hassan Hatefi, Holger Hermanns, Jan Krˇ c´ al Saarland University – Computer Science, Saarbr¨ ucken, Germany May 6, 2015 Yuliya Butkova, Hassan Hatefi, Holger Hermanns, Jan Krˇ c´ al Continuous Optimal Timing
Motivation Definitions and Problem Statement Existing Algorithms Our Algorithm Empirical Evaluation Conclusion Outline Motivation Preliminaries Existing Algorithms Our Algorithm Empirical Evaluation Conclusion Yuliya Butkova, Hassan Hatefi, Holger Hermanns, Jan Krˇ c´ al Continuous Optimal Timing
Motivation Definitions and Problem Statement Existing Algorithms Our Algorithm Empirical Evaluation Conclusion Motivation Probabilistic models – unreliable/unpredictable system behaviour: message loss, component failure, ... – randomized algorithms: the probability of reaching consensus in leader election algorithms is almost 1 Yuliya Butkova, Hassan Hatefi, Holger Hermanns, Jan Krˇ c´ al Continuous Optimal Timing
Motivation Definitions and Problem Statement Existing Algorithms Our Algorithm Empirical Evaluation Conclusion Motivation Models we work with: – run in continuous time – comprise non-deterministic and probabilistic behaviour are good for: – optimization over multiple available choices – finding worst case results properties: Is the maximal probability of reaching a failure state within an hour < 0.01? Yuliya Butkova, Hassan Hatefi, Holger Hermanns, Jan Krˇ c´ al Continuous Optimal Timing
Motivation Definitions and Problem Statement Existing Algorithms Our Algorithm Empirical Evaluation Conclusion Motivation Model checking boils down to time-bounded reachability problem: What is the maximal/minimal probability to reach a given set of states within a given time bound? Several algorithms to tackle this problem are known they are polynomial, but still slow on industrial size benchmarks there is no proper comparison between all of them no one has a clue which algorithm will be faster on a specific benchmark Yuliya Butkova, Hassan Hatefi, Holger Hermanns, Jan Krˇ c´ al Continuous Optimal Timing
Motivation Definitions and Problem Statement Existing Algorithms Our Algorithm Empirical Evaluation Conclusion Outline Motivation Preliminaries Existing Algorithms Our Algorithm Empirical Evaluation Conclusion Yuliya Butkova, Hassan Hatefi, Holger Hermanns, Jan Krˇ c´ al Continuous Optimal Timing
Motivation Definitions and Problem Statement Existing Algorithms Our Algorithm Empirical Evaluation Conclusion CTMDPs have some money waste 10 reliable risky broke 1 1 99 gamble do a PhD a a 1 0 0 1 0 . 0 rich Yuliya Butkova, Hassan Hatefi, Holger Hermanns, Jan Krˇ c´ al Continuous Optimal Timing
Motivation Definitions and Problem Statement Existing Algorithms Our Algorithm Empirical Evaluation Conclusion CTMDPs Continuous Time Markov Decision Process have some money (CTMDP) waste is a tuple C = ( S , Act , ❘ ), where 10 – S - set of states reliable risky broke – Act - set of actions – ❘ : S × Act × S �→ R ≥ 0 rate function 1 1 99 gamble do a PhD a a 1 0 0 1 0 . 0 rich Yuliya Butkova, Hassan Hatefi, Holger Hermanns, Jan Krˇ c´ al Continuous Optimal Timing
Motivation Definitions and Problem Statement Existing Algorithms Our Algorithm Empirical Evaluation Conclusion CTMDPs Continuous Time Markov Decision Process have some money (CTMDP) waste is a tuple C = ( S , Act , ❘ ), where 10 – S - set of states reliable risky broke – Act - set of actions – ❘ : S × Act × S �→ R ≥ 0 rate function 1 1 99 gamble do a PhD Exit Rate E ( s , α ) = � ❘ ( s , α, s ′ ) s ′ ∈ S a a CTMDP is Uniform if exit rates over all 1 0 0 1 states and all available actions are the same 0 . 0 rich Yuliya Butkova, Hassan Hatefi, Holger Hermanns, Jan Krˇ c´ al Continuous Optimal Timing
Motivation Definitions and Problem Statement Existing Algorithms Our Algorithm Empirical Evaluation Conclusion Resolution of Non-Determinism. Schedulers. have some money What is the probability of becoming reach waste before I die? 10 reliable risky broke 1 1 99 gamble do a PhD a a 1 0 0 1 0 . 0 rich Yuliya Butkova, Hassan Hatefi, Holger Hermanns, Jan Krˇ c´ al Continuous Optimal Timing
Motivation Definitions and Problem Statement Existing Algorithms Our Algorithm Empirical Evaluation Conclusion Resolution of Non-Determinism. Schedulers. have some money What is the probability of becoming reach waste before I die? 10 The answer depends on chosen actions reliable risky broke 1 1 99 gamble do a PhD a a 1 0 0 1 0 . 0 rich Yuliya Butkova, Hassan Hatefi, Holger Hermanns, Jan Krˇ c´ al Continuous Optimal Timing
Motivation Definitions and Problem Statement Existing Algorithms Our Algorithm Empirical Evaluation Conclusion Resolution of Non-Determinism. Schedulers. have some money What is the probability of becoming reach waste before I die? 10 The answer depends on chosen actions reliable risky broke A Scheduler σ (or controller, policy ): 1 1 σ : History → Act 99 gamble do a PhD Classes of schedulers: a a Timed/Untimed - knowledge of time passed ( Tim/Unt ) 1 0 0 1 0 . Early/Late - decision is fixed on 0 rich entering a state/maybe changed at any time later Yuliya Butkova, Hassan Hatefi, Holger Hermanns, Jan Krˇ c´ al Continuous Optimal Timing
Motivation Definitions and Problem Statement Existing Algorithms Our Algorithm Empirical Evaluation Conclusion Reachability Problem have some money waste 10 What is the maximal/minimal probability to reach a given set of states within given time? reliable risky broke val ∇ ( s ) := Pr s ♦ ≤ T G � � sup σ σ ∈ Tim ∇ 1 1 ∇ ∈ { ℓ, e } 99 gamble do a PhD a a 1 0 0 1 0 . 0 rich Yuliya Butkova, Hassan Hatefi, Holger Hermanns, Jan Krˇ c´ al Continuous Optimal Timing
Motivation Definitions and Problem Statement Existing Algorithms Our Algorithm Empirical Evaluation Conclusion Outline Motivation Preliminaries Existing Algorithms Our Algorithm Empirical Evaluation Conclusion Yuliya Butkova, Hassan Hatefi, Holger Hermanns, Jan Krˇ c´ al Continuous Optimal Timing
Motivation Definitions and Problem Statement Existing Algorithms Our Algorithm Empirical Evaluation Conclusion Existing Algorithms Early Late Exponential Approximation Polynomial Approximation ExpStep -1 PolyStep -k (by M. Neuhaeussar, L. Zhang) (by J. Fearnley, M. Rabe, et al.) Improved Exponential Adaptive Step Approximation Approximation AdaptStep ExpStep -k (by P. Buchholz, I. Schulz) (by H. Hatefi, H. Hermanns) All existing approaches use discretization Yuliya Butkova, Hassan Hatefi, Holger Hermanns, Jan Krˇ c´ al Continuous Optimal Timing
Motivation Definitions and Problem Statement Existing Algorithms Our Algorithm Empirical Evaluation Conclusion Outline Motivation Preliminaries Existing Algorithms Our Algorithm Empirical Evaluation Conclusion Yuliya Butkova, Hassan Hatefi, Holger Hermanns, Jan Krˇ c´ al Continuous Optimal Timing
Motivation Definitions and Problem Statement Existing Algorithms Our Algorithm Empirical Evaluation Conclusion Our Approach Features: Does NOT discretize the time horizon, instead approximate via different class of schedulers: Less powerfull Untimed - for lower bound More powerfull “Prophetic” - for upper bound Yuliya Butkova, Hassan Hatefi, Holger Hermanns, Jan Krˇ c´ al Continuous Optimal Timing
Motivation Definitions and Problem Statement Existing Algorithms Our Algorithm Empirical Evaluation Conclusion Our Algorithm ( Unif + ) input : CTMDP C = ( S , Act , R ), goal states G ⊆ S , horizon T ∈ R > 0 , scheduler class ∇ ∈ { ℓ, e } , and approximation error ε > 0 params : truncation error ratio κ ∈ (0 , 1) output : vector v such that � v − val ∇ � ∞ ≤ ε 1 λ ← maximal exit rate E max in C 2 repeat C ∇ λ ← ∇ -uniformisation of C to the rate λ 3 v ← approximation of the lower bound val for C ∇ λ up to error ε · κ 4 v ← approximation of the upper bound val for C ∇ λ up to error ε · κ 5 λ ← 2 · λ 6 7 until � v − v � ∞ ≤ ε · (1 − κ ) 8 return v Yuliya Butkova, Hassan Hatefi, Holger Hermanns, Jan Krˇ c´ al Continuous Optimal Timing
Motivation Definitions and Problem Statement Existing Algorithms Our Algorithm Empirical Evaluation Conclusion Our Algorithm ( Unif + ) input : CTMDP C = ( S , Act , R ), goal states G ⊆ S , horizon T ∈ R > 0 , scheduler class ∇ ∈ { ℓ, e } , and approximation error ε > 0 params : truncation error ratio κ ∈ (0 , 1) output : vector v such that � v − val ∇ � ∞ ≤ ε 1 λ ← maximal exit rate E max in C 2 repeat C ∇ λ ← ∇ -uniformisation of C to the rate λ 3 v ← approximation of the lower bound val for C ∇ λ up to error ε · κ 4 v ← approximation of the upper bound val for C ∇ λ up to error ε · κ 5 λ ← 2 · λ 6 7 until � v − v � ∞ ≤ ε · (1 − κ ) 8 return v Yuliya Butkova, Hassan Hatefi, Holger Hermanns, Jan Krˇ c´ al Continuous Optimal Timing
Recommend
More recommend
