A Storm is Coming! A New Probabilistic Model Checker Joost-Pieter - PowerPoint PPT Presentation

A Storm is Coming! A New Probabilistic Model Checker Joost-Pieter Katoen IFIP WG 1.8 Open Problems in Concurrency Theory, June 26, 2017 joint with: Christian Dehnert, Sebastian Junges and Matthias Volk

Probabilistic Model Checking Theory in practice for system design and verification. ACM SIGLOG News, 3, 2015 “A promising new direction in formal methods research these days is the development of probabilistic models, with associated tools for quantitative evaluation of system performance along with correctness” “probabilistic model checking is one of the main challenges for the future” The birth of model checking. 2008 2

Probabilistic Model Checking First approaches soon after the birth of model checking mostly focused on almost-sure events Second generation focused on quantitative properties omega-regular events, probabilistic CTL Since the early 2000s, powerful model checkers exist Bright future: many application areas, e.g. robotics 3

Probabilistic Model Checkers Model Checkers: Applications: > 10,000 HVC Award downloads PRISM 2016 Reliability Engineering MRMC Performance Evaluation LiQuor Dependability Analysis iscasMC Systems Biology Robotics MoDeST M. Kwiatkowska Software Engineering MARCIE Model Repair GreatSPN IMCA ……… PASS New: the PARAM STORM D.Parker G. Norman ……… model checker plus all statistical “model checkers” 4

This Talk A. The ins and outs of the STORM model checker B. STORM’s performance compared to PRISM C. STORM’s performance compared to other competitors D. STORM’s support for Markov automata with multiple objectives 5

STORM’s Characteristics It supports several native input languages Models: Markov chains and MDPs, and Markov automata Supports explicit state, fully symbolic and hybrid engines It has a modular set-up: easy exchange of solvers currently: 15 solvers, CUDD and Sylvan 6

STORM’s Characteristics Supports a Python interface for rapid prototyping Hosts many functionalities under a single roof: (high-level) counterexample synthesis permissive scheduler synthesis conditional probabilities and rewards game-based abstraction of infinite-state MDPs long-run averages on MDPs Mostly faster than all competitors 7

STORM’s Architecture STORM comprises about 100,000 C++ code lines 8

STORM’s Solvers 9

STORM’s Input Languages — Probabilistic Programs cp := [0 , . . . , 0]; i := 1; x := N ; C ccp : while ( x > 0) { while ( cp [ i ] 6 = 0) { i : ⇡ Unif [1 . . . N ] } ; cp [ i ] := 1; x := x � 1 } Programs in probabilistic GCL + observe-statements 
 Automated abstraction techniques for unbounded variables 
 Used in security, machine learning, AI, etc. 10

STORM’s Input Languages — Dynamic Fault Trees Dugan’s DFTs with, p-FDEPs, nested SPAREs 
 Tailored state-space generation and reduction techniques 
 One of the—if not the — most prominent models in reliability engineering 11

STORM’s Input Languages — Generalised Stochastic Petri Nets Petri Nets with “exponential” 
 and “immediate” transitions Supports confused GSPNs 
 , E I C R A M n i t o . n c t e r t a m S , N P S t a e r G One of the—if not the — most 
 prominent models in performance 
 and dependability analysis 12

This Talk A. The ins and outs of the STORM model checker B. STORM’s performance compared to PRISM C. STORM’s performance compared to other competitors D. STORM’s support for Markov automata with multiple objectives 13

Performance Comparison with PRISM Compare best engines (left) and exact arithmetic engines (right) 14

Performance Comparison with PRISM All PRISM benchmark models with all 84 properties Compare engines that are conceptually similar 
 8-core proc (2.0 GHz) with 8GB RAM; timeout = 1800 s 15

This Talk A. The ins and outs of the STORM model checker B. STORM’s performance compared to PRISM C. STORM’s performance compared to other competitors D. STORM’s support for Markov automata with multiple objectives 16

How Many Problems Can be Solved in Time? Compare best engines of EPMC, PRISM and STORM 17

Markov Automata 18

STORM’s Performance on Markov Automata Compare IMCA against STORM (sparse) on all IMCA models Reachability, expected rewards, and long-run rewards Time-bounded and reward-bounded reachability 19

No STORM Engine Prevails 20

STORM’s DFT State Space Generation Monolithic state-space generation Don’t care propagation, symmetry and partial-order reduction Modularisation: analyse independent sub-DFTs separately 21

Performance of DFT Analysis 22

Performance of Parameter Synthesis Comparison to PARAM and PRISM on parametric Markov chains 23

This Talk A. The ins and outs of the STORM model checker B. STORM’s performance compared to PRISM C. STORM’s performance compared to other competitors D. STORM’s support for Markov automata with multiple objectives 24

Outlook: Markov Automata with Multiple Objectives Stochastic job-shop scheduling schedule n jobs on k machines under pre-emptive scheduling each job has an exponential duration LEPT scheduling optimal to minimise expected completion time How to schedule if multiple constraints are imposed? the expected completion time of all jobs below a threshold and finish 50% of all jobs quickly too. Trade-off! Pareto 25

STORM’s Performance 26

STORM’s Performance Comparison 27

Take-Home Message STORM is modular. STORM is extendible. STORM is fast(er). https://stormchecker.org/ 28