Ivy: Interactive Verification of Parameterized Systems (via Effectively Propositional Reasoning) Oded Padon Joint work with: Ken McMillan and Mooly Sagiv Dagstuhl Seminar, 3-Nov-2015
Motivation • Many real-world verification efforts use little proof automation: • CompCert, seL4, Verdi, IronClad/IronFleet, Intel, AMD • Automated tools are brittle and opaque • Fail unpredictably and completely • Unavoidable since problems are intractable / undecidable • Diagnosing and correcting failures is hard because tools do not effectively communicate with the user • With interactive tools, the system always makes progress • Goal: Put the user “in the loop” of more automated verification • Use powerful invariant generation heuristics interactively • Bidirectional feedback between user and machine heuristics
Effectively Propositional Logic – EPR a.k.a. Bernays-Schönfinkel-Ramsey class • Fragment of first-order logic • Restricted quantifier prefix: * * φ Q.F. • No function symbols • Small model property • x 1 ,…, x n . y 1 ,…, y m . φ Q.F. has a model iff it has a model of at most n+k elements (k - number of constant symbols) • Satisfiability is decidable • Support from Z3 F. Ramsey. On a problem in formal logic. Proc. London Math. Soc. 1930
Using EPR for Verification • I is an inductive invariant iff I ∧ TR ∧ ¬I ’ is unsatisfiable • EPR transition system: • States – finite structures • Transition relation – two vocabulary * * formula • Checking inductiveness of alternation-free invariants is decidable • Expressible in EPR: • Partial function • Injective function • Partial order • Total order • Not expressible in EPR: • Total function • Surjective function
Prior Work on EPR based Verification • Itzhaky et al. showed EPR can be used to reason about reachabilty in linked lists by using n* instead of n • Karbyshev et al. showed a version of IC3/PDR that infers universal invariants for EPR transition systems • Padon et al. showed that under certain conditions (e.g. linked lists) inferring universal inductive invariants is decidable S. Itzhaky et al. Effectively-Propositional Reasoning about Reachability in Linked Data Structures . CAV 2013 S. Itzhaky et al. Modular reasoning about heap paths via effectively propositional formulas. POPL 2014 A. Karbyshev et al. Property-Directed Inference of Universal Invariants or Proving Their Absence. CAV 2015 O. Padon et al. Decidability of Inferring Inductive Invariants. POPL 2016
Ivy Overview Elements of Interactive Verification • EPR based verification • Graphical visualization of proof state and proof failures • Visualization of sets of structures - concept graph • Visualization of carefully chosen concrete transitions • Web-based UI (embedded in IPython/Jupyter notebook) • Interaction modes • Bounded verification with BMC • Abstract reachability graph based • Counterexample to induction based • Scriptable interface
Ivy Overview Elements of Interactive Verification • EPR based verification • Graphical visualization of proof state and proof failures • Visualization of sets of structures - concept graph • Visu isuali lization of of car arefull lly ch chosen con oncrete tr tran ansit itions • Web-based UI (embedded in IPython/Jupyter notebook) • Interaction modes • Bounded verification with BMC • Abstract reachability graph based • Cou ounterexample le to o in induction base ased • Scriptable interface
CTI based Interaction Mode Candidate Inductive Invariant Model Yes Inductive Invariant Found Inductive? Modify candidate invariant No Display “ minimal” CTI Diagnose CTI User Heuristics
CTI based Interaction Mode Candidate Inductive Invariant Model Yes Inductive Invariant Found Inductive? Modify candidate invariant No Z3 (EPR) Display “ minimal” CTI Diagnose CTI User Heuristics
Example: Leader Election in a Ring 3 5 • Nodes are organized in a ring • Each node has a unique numeric id 1 2 • Protocol: • Each node sends its id to the left 4 6 • A node that receives a massage passes it (to the left) if the id in the message is higher than the node’s own id • A node that receives its own id becomes the leader • Theorem: • The protocol selects at most one leader E. Chang and R. Roberts. An improved algorithm for decentralized extrema-finding in circular configurations of processes . CACM 1979
Example: Leader Election in a Ring 3 5 • Nodes are organized in a ring • Each node has a unique numeric id 1 2 • Protocol: • Each node sends its id to the left 4 6 • A node that receives a massage passes it (to the left) if the id in the message is higher than the node’s own id • A node that receives its own id becomes the leader • Theorem: • The protocol selects at most one leader E. Chang and R. Roberts. An improved algorithm for decentralized extrema-finding in circular configurations of processes . CACM 1979
Leader Protocol Model • Sorts: Node, ID • Constants: • rin ring_head: Node • rin ring_tail il: Node • zero: ID • le lead ader_id id: ID • Relations: • le le(ID, ID) – total order on node id’s • reach(Node, Node) – total order, describes ring topology • id id(Node, ID) – relate a node to its id • pendin ing(ID, Node) – models pending messages • le lead ader(Node) – leader(n) means n is the leader
Leader Protocol Model action send: action receive: m, n1 := pending.nd_pop() n1 := * i1 := id(n1) n2 := next_in_ring(n1) if i1 = m: i1 := id(n1) # found leader leader .insert(n1) pending.insert(i1, n2) leader_id := i1 else if le(i1, m): # pass message n2 := next_in_ring(n1) pending.insert(m, n2) • le le(ID, ID) – total order on node id’s • reach(Node, Node) – total order, describes ring topology • id id(Node, ID) – relate a node to its id • pen endin ing(ID, Node) – models pending messages • lea leader(Node) – leader(n) means n is the leader
Leader Protocol Model action send: action receive: m, n1 := pending.nd_pop() n1 := * i1 := id(n1) n2 := next_in_ring(n1) if i1 = m: i1 := id(n1) # found leader leader .insert(n1) pending.insert(i1, n2) leader_id := i1 else if le(i1, m): # pass message n2 := next_in_ring(n1) pending.insert(m, n2) init ¬ leader (N) ∧ ¬pending(I,N) ∧ leader_id = zero conjecture ( leader (X) ∧ leader (Y)) X = Y conjecture leader (X) -> id(X, leader_id ) conjecture (id(X, I) ∧ id(Y, J) ∧ leader (X)) le(J, I)
DEMO
Heuristics for User Interaction • CTI CTI min inim imization Model Invariant • Small number of elements Yes • “Small” relations according to Inductive? user’s choice / model hints Modify Invariant • Dia Diagnosis is in interactions No • Select visible relations Display “ minimal” CTI • Gather facts from user selection • BMC • Check conjecture Diagnose CTI • Minimize conjecture User Heuristics • Sufficiency for current failure • Relative inductiveness
Observations • User intuition and machine heuristics complement each other: • User has better bias towards relevant and inductive facts that leads to better generalizations • Machine is better at finding bugs and corner cases • Challenge is finding useful interaction mode and heuristics • Might not be the same as fully automated heuristics (e.g. PDR) • Heuristics will sometimes fail, so must leave a “manual” mode
Observations • User intuition and machine heuristics complement each other: • User has better bias towards relevant and inductive facts that leads to better generalizations • Machine is better at finding bugs and corner cases • Challenge is finding useful interaction mode and heuristics • Might not be the same as fully automated heuristics (e.g. PDR) • Heuristics will sometimes fail, so must leave a “manual” mode • EPR allows predictability of automated analysis • EPR small model property makes BMC more powerful • Bound is only on TR unrolling, not on universe size • EPR verification sometimes requires additional ghost state
Applications Application Status Distributed spanning tree Modeled and verified Network learning switch Modeled and verified Leader in Ring Modeled and verified Chord Modeled and verified Flash Modeled and verified* Copy Garbage Collection Modeled and verified* Bakery Algorithm Modeled, work in progress Work in progress Distributed Database (with Aurojit Panda, UC Berkeley) We are looking for more applications, please let me know your suggestions! *Not yet full functional correctness, still work in progress
Ivy as a Research Framework • Ivy is a framework for exploring interactive verification: • EPR based verification using Z3 • Graphical visualization (concrete structures, concept graphs) • Web-based UI (embedded in IPython/Jupyter notebook) • Scriptable interface
Ivy as a Research Framework • Ivy is a framework for exploring interactive verification: • EPR based verification using Z3 • Graphical visualization (concrete structures, concept graphs) • Web-based UI (embedded in IPython/Jupyter notebook) • Scriptable interface CTI Mode Inv M Ind? Modify Inv “ minimal” CTI Diagnose CTI User Heuristics
Recommend
More recommend
