MODEL DATA PREDICTION by VERIFICATION I NTRODUCTION B ACKGROUND S - PowerPoint PPT Presentation

M ACHINE L EARNING MEETS F ORMAL V ERIFICATION Luca Bortolussi 1 , 2 1 Dipartimento di Matematica e Geoscienze Università degli studi di Trieste, Italy 2 Modelling and Simulation Group Saarland University, Saarbücken, Germany GANDALF, September 14, Catania, Italy

I NTRODUCTION B ACKGROUND S MOOTH MC R OBUST D ESIGN C ONCLUSIONS 2 / 60 C OMPLEX S YSTEMS hex

I NTRODUCTION B ACKGROUND S MOOTH MC R OBUST D ESIGN C ONCLUSIONS 3 / 60 M ODELLING C OMPLEX S YSTEMS MODEL DATA PREDICTION by VERIFICATION

I NTRODUCTION B ACKGROUND S MOOTH MC R OBUST D ESIGN C ONCLUSIONS 4 / 60 M ODELLING C OMPLEX S YSTEMS MODEL DATA UNCERTAINTY PREDICTION by VERIFICATION

I NTRODUCTION B ACKGROUND S MOOTH MC R OBUST D ESIGN C ONCLUSIONS 5 / 60 M ODELLING C OMPLEX S YSTEMS MODEL DATA UNCERTAINTY l m a m a c r s h o d f l i e n o a e h r t e n i m n g PREDICTION by VERIFICATION

I NTRODUCTION B ACKGROUND S MOOTH MC R OBUST D ESIGN C ONCLUSIONS 6 / 60 T HE “ PHILOSOPHICAL ” POINT F ORMAL M ETHODS Quantitative Formal methods provide (scalable) tools to deal with both local (e.g. Stochastic Process Algebras) and global aspects (e.g. temporal logic), which are seamlessly integrated (through stochastic verification). M ACHINE L EARNING Machine Learning provides us with sophisticated statistical tools to manage and refine uncertainty, in a computationally efficient way. M ANTRA Integrate machine learning with quantitative formal methods to produce verification tools dealing consistently with uncertainty.

I NTRODUCTION B ACKGROUND S MOOTH MC R OBUST D ESIGN C ONCLUSIONS 7 / 60 T HE PROBLEMS TACKLED ( SO FAR ) Satisfaction under Uncertainty : Compute satisfaction (probabilities) of formal properties for uncertain models. Synthesis and Design : Find model parameters satisfying (as robustly as possible) a set of formal specifications. Parameter Estimation : Learn model parameters from qualitative observations. Model simplification : Learn simplified models and correction maps. Control : Learn optimal control policies in MDPs.

I NTRODUCTION B ACKGROUND S MOOTH MC R OBUST D ESIGN C ONCLUSIONS 8 / 60 B IBLIOGRAPHY L. Bortolussi, D. Milios, G. Sanguinetti. Smoothed model checking for uncertain Continuous-Time Markov Chains. Information and Computation, 2016. L. Bortolussi, D. Milios, G. Sanguinetti. U-Check: Model Checking and Parameter Synthesis Under Uncertainty. QEST 2015. E. Bartocci, L. Bortolussi, L. Nenzi, G. Sanguinetti. System design of stochastic models using robustness of temporal properties. Theoretical Computer Science, 2015. E. Bartocci, L. Bortolussi, D. Milios, L. Nenzi, G. Sanguinetti. Studying Emergent Behaviours in Morphogenesis Using Signal Spatio-Temporal Logic. HSB, 2015. L. Bortolussi, G. Sanguinetti. Learning and Designing Stochastic Processes from Logical Constraints. Logical Methods in Computer Science, 2015. L. Bortolussi, D. Milios, G. Sanguinetti. Efficient Stochastic Simulation of Systems with Multiple Time Scales via Statistical Abstraction. CMSB 2015 L. Bortolussi, G. Caravagna, G. Sanguinetti. Matching Models Across Abstraction Levels with Gaussian Processes. CMSB 2016. E. Bartocci, L. Bortolussi, T. Bradzil, D. Milios, G. Sanguinetti. Policy Learning for Time-Bounded Reachability in Continuous-Time Markov Decision Processes via Doubly-Stochastic Gradient Ascent. QEST 2016.

I NTRODUCTION B ACKGROUND S MOOTH MC R OBUST D ESIGN C ONCLUSIONS 9 / 60 O UTLINE OF THE TALK Quick background on stochastic modelling and formal methods Gaussian Processes and Smoothed Model Checking (Robust System Design)

O UTLINE 1 I NTRODUCTION 2 B ACKGROUND 3 S MOOTHED MODEL CHECKING 4 R OBUST D ESIGN 5 C ONCLUSIONS

I NTRODUCTION B ACKGROUND S MOOTH MC R OBUST D ESIGN C ONCLUSIONS 11 / 60 A SIMPLE EXAMPLE : EPIDEMIC SPREADING We consider a simple model of S epidemic spreading in a ext homogeneous network (with average degree ⟨ k ⟩ ). inf I R Each individual can be in three recover states: susceptible ( S ), infected ( I ), and recovered ( R ). S ext loss The state of an agent can change inf I R patch due to three events: S S ext ext loss loss inf inf I R I R (infection) I + S → I + I patch patch 1 at rate k i ⟨ k ⟩ SI (ext infection) S → I 2 S S ext ext loss loss inf inf at rate k e S I R I R patch patch S ext (recovery) I → R loss 3 inf I R patch at rate k r I

I NTRODUCTION B ACKGROUND S MOOTH MC R OBUST D ESIGN C ONCLUSIONS 12 / 60 T HE STOCHASTIC PROCESS BEHIND : CTMC We consider population CTMC models, that describe the dynamics of a population of interacting agents. S TATE SPACE The state space is described by a set of n variables X = X 1 ,..., X n ∈ N , each counting the number of agents (jobs, molecules, ...) of a given kind. T RANSITIONS The dynamics is given by a set of rules of the form s 1 X 1 + ... + s n X n → r 1 X 1 + ... + r n X n , with rate given by a function f ( X ,θ ) , depending on the system variables and on a set of parameters θ .

I NTRODUCTION B ACKGROUND S MOOTH MC R OBUST D ESIGN C ONCLUSIONS 13 / 60 T HE STOCHASTIC PROCESS BEHIND : CTMC R ULES s 1 X 1 + ... + s n X n → r 1 X 1 + ... + r n X n Update vector: v = ( r 1 − s 1 ,..., r n − s n ) Rate: f ( X ,θ ) . M ASTER E QUATION d P ( X , t ) = ∑ P ( X − v j , t ) f ( X − v j ,θ ) d t − ∑ P ( X , t ) f ( X ,θ ) d t j j S IMULATION We can simulate such model with standard algorithms, the most known is Gillespie’s one.

I NTRODUCTION B ACKGROUND S MOOTH MC R OBUST D ESIGN C ONCLUSIONS 14 / 60 A SIMPLE EXAMPLE : EPIDEMIC SPREADING We are often interested in qualitative features of the system, such as S ext loss inf I R The epidemics stopped 1 patch S S between days 60 and 90. ext ext loss loss inf inf I R I R patch patch The final fraction of infected 2 people is between 80% and 90%. S S ext ext loss loss inf inf The fraction of infected people 3 I R I R patch patch S ext remained below 45%. loss inf I R patch We will formalise them with metric interval temporal logic.

I NTRODUCTION B ACKGROUND S MOOTH MC R OBUST D ESIGN C ONCLUSIONS 15 / 60 M ETRIC I NTERVAL T EMPORAL L OGIC We express qualitative properties by Metric (interval) Temporal Logic. Linear continuous time: in experiments we observe single realisations. Metric bounds: we can observe a system only for a finite amount of time. S YNTAX ϕ ∶∶ = tt ∣ µ ∣ ¬ ϕ ∣ ϕ 1 ∧ ϕ 2 ∣ ϕ 1 U [ T 1 , T 2 ] ϕ 2 , As customary: F [ T 1 , T 2 ] ϕ ≡ tt U [ T 1 , T 2 ] ϕ , G [ T 1 , T 2 ] ϕ ≡ ¬ F [ T 1 , T 2 ] ¬ ϕ . F [ T 1 , T 2 ] ϕ : ϕ is eventually true between time T 1 and T 2 . G [ T 1 , T 2 ] ϕ : ϕ is always true between time T 1 and T 2 . ϕ 1 U [ T 1 , T 2 ] ϕ 2 : ϕ 1 is true until ϕ 2 becomes true between time T 1 and T 2 .

I NTRODUCTION B ACKGROUND S MOOTH MC R OBUST D ESIGN C ONCLUSIONS 16 / 60 MITL FOR EPIDEMIC SPREADING G [ 0 , 200 ] ( X I < 45 ) the fraction of infected never exceeds 45% in the S ext loss inf first 200 time units. This upper bounds the I R patch number of active infecting indivuals. S S ext ext loss loss inf inf I R I R patch patch F [ 22 , 40 ] ( X I > 35 ) S S ext ext loss loss inf inf I R I R patch patch between time 22 and 40, the fraction of infected S ext loss inf I R patch exceeds 35%. This locates the infection peak between time 22 and 40. ( X I > 0 ) U [ 100 , 120 ] ( X I = 0 ) the epidemic stops between time 100 and 120. G [ 90 , 200 ] ( 82 < X R < 88 ) the final fraction of recovered is between 82% and 88%. This corresponds to the fraction of population which was infected.

I NTRODUCTION B ACKGROUND S MOOTH MC R OBUST D ESIGN C ONCLUSIONS 17 / 60 T HE MODEL CHECKING PROBLEM Given a MiTL formula ϕ , and a trajectory σ , we can algorithmically check if ϕ holds in the world described by σ (starting at time zero). In this case, we say that σ is a model of ϕ , and we write σ ⊧ ϕ . A stochastic model defines a probability distribution on the space of trajectories. In this case, given a MiTL formula ϕ , we are interested in the probability of the subset of trajectories that satisfies ϕ : p ( ϕ ) = p { X = σ ∣ σ ⊧ ϕ } The stochastic model checking problem is thus the problem of computing p ( ϕ ) , for a given stochastic process X ( t ) .

I NTRODUCTION B ACKGROUND S MOOTH MC R OBUST D ESIGN C ONCLUSIONS 18 / 60 S TATISTICAL M ODEL C HECKING We start with a CTMC with fixed parameters θ and a MiTL formula ϕ . We generate samples of the Bernoulli random variable Z ϕ , equal to 1 if and only if ϕ is true as follows: generate a sample trajectory of the CTMC, e.g. by SSA algorithm run a monitoring algorithm on the trajectory to establish the truth of ϕ . Use statistical tools (Wald sequential testing, Bayesian approach) to establish if p ( ϕ ∣ θ ) > q is true (with confidence level α ) or to estimate p ( ϕ ∣ θ ) .

O UTLINE 1 I NTRODUCTION 2 B ACKGROUND 3 S MOOTHED MODEL CHECKING 4 R OBUST D ESIGN 5 C ONCLUSIONS