Symbolic Abstract Data Types (ADTs) Constantin Enea University - PowerPoint PPT Presentation

Symbolic Abstract Data Types (ADTs) Constantin Enea University Paris Diderot - Paris 7 joint work with Michael Emmi, IMDEA Madrid

Modular Reasoning Abstract Data Types smaller & simpler program, easier to reason about ⊧ 𝜒 ⊧ ⊧ Compositional Proof Rule your 𝜒 program ⊧ that could be checked once for all programs complex objects

Concurrent objects • methods are called from different threads • behaviors = sequences of method calls and returns call t1 push(1) ret t1 OK call t2 pop() call t1 pop() ret t1 1 …. ret t2 3 • • more abstractly, as histories (posets of invocations) happens-before pop ⇒ 3 partial order push(1) pop ⇒ 2 pop ⇒ 1 push(2) push(3)

ADTs, Linearizability • ADT = set of sequential histories (total orders) O |= ADT iff ∀ h ∈ Histories(O) ∃ linearization ∈ ADT ∉ push(1) pop ⇒ 3 pop ⇒ 1 push(2) push(3) pop ⇒ 2 pop ⇒ 3 push(1) pop ⇒ 2 ∉ push(1) pop ⇒ 1 pop ⇒ 3 push(2) push(3) pop ⇒ 2 pop ⇒ 1 push(2) push(3) ∃ linearization ∈ SeqStack ? ∉ push(1) pop ⇒ 1 push(2) pop ⇒ 3 push(3) pop ⇒ 2 ∈ push(1) pop ⇒ 1 push(2) push(3) pop ⇒ 3 pop ⇒ 2

Modular Reasoning 1. Is there a valid linearization for each history ? 2. Does every valid linearization preserve client invariants ? Challenges: • a history has exponentially-many linearizations (checking linearizability is more complex than checking reachability) • infinite sets of linearizations Approach: symbolic representations of seq. hist. (linearizations) (first order logic)

Describing ADTs • declarative specifications rather than state-based • specifying a register: every read returns the value written by the most recent write ∀ o1, o2, o3. “o1 is a write(a)” ∧ “o2 is a write(b)” ∧ “o3 is a read=>a” ∧ o1 < o2 => o3 < o2

Inferring Symbolic ADTs [Emmi, E , POPL’16] GOAL 
 inferring FO descriptions of ADTs HYPOTHESIS 
 • violations can be decomposed in a small set of patterns • patterns manifest with few operations

Challenges • infinite sets of method invocations and input/output values • characterizing infinite sets of violations with a finite set of patterns : • adding ops. to violations may lead to correct hist. violation correct pop ⇒ 1 push(1) pop ⇒ 1

Abstracting Data: Matchings • Group operations in matchings (operations referring to the same instances of values) pop ⇒ 2 pop ⇒ 1 push(1) push(2) pop ⇒ EMPTY deq ⇒ 2 enq(1) enq(2) deq ⇒ 1 • Describing histories in first-order logic: • Empty: an unmatched push cannot precede a pop ⇒ EMPTY • Fifo order: for every two ordered enqueues, the matching dequeues cannot be in the reverse order

Abstract Algorithm Input: a reference implementation Impl Output: FO formula representing Histories(Impl) patterns = O ; for each seq. hist. h do if * then break; else if h is executable with Impl then continue; else if h is redundant with patterns then continue; else add h to patterns return exclusion of patterns

Example: atomic queue One deq per enq dequeuing empty deq ⇒ 1 enq(1) deq ⇒ EMPTY deq ⇒ 1 enq(1) deq ⇒ EMPTY enq(1) deq ⇒ 1 deq ⇒ 1 enq(1) deq ⇒ 1 fifo order deq ⇒ 2 deq ⇒ 2 enq(1) enq(2) enq(1) enq(2) deq ⇒ 1

Abstract Algorithm Input: a reference implementation Impl Output: FO formula representing Histories(Impl) patterns = O ; for each seq. hist. h do if * then break; else if h is executable with Impl then continue; else if h is redundant with patterns then continue; else add h to patterns return exclusion of patterns some pattern ⪯ h

History Weakening push(1) pop ⇒ 1 push(2) pop ⇒ 2 pop ⇒ EMPTY pop ⇒ EMPTY all concurrent weaker order push(1) pop ⇒ 1 push(2) pop ⇒ 2 LEMMA 
 Libraries closed under weakening • “weaker than” = linearizability

Closure properties • LEMMA. Libraries closed under weakening • Closure under removing matches pop ⇒ 2 pop ⇒ EMPTY push(1) pop ⇒ 1 push(2) • Closure under removing duplicates write(1) read ⇒ 1 read ⇒ 1 read ⇒ 1

Weaker than ++ h ⪯ h’ iff h is weaker than h’ or h has more matches or duplicates PROPOSITION 
 Typical concurrent libraries are closed under ⪰ • the set of violations is closed under ⪯ THEOREM 
 ⪯ is a well quasi-order on bounded-width histories bounded number of threads

Inference Algorithm • enumerate histories according to their size • patterns = minimal elements w.r.t. ⪯ • closure properties => all histories greater than patterns are violations • ⪯ is a wqo => there are only finitely-many patterns • stop , when increasing the size doesn’t add new patterns • typically, at most 4 ops. suffice to infer complete specifications

Excluding Patterns • formulas describing patterns where operations are existentially-quantified deq ⇒ 1 enq(1) ∃ o, o’: lab (o) = deq ⇒ 1 ⋀ lab (o’) = enq(1) ⋀ o < o’ ⋀ match (o) = o’ • the negation describes all histories greater than the pattern

Runtime Monitoring [Emmi, E , Hamza, PLDI’15] GOAL 
 online history checking 
 • complete per sample 
 • constant space o/w: exhaust memory 
 • linear time o/w: progressive slowdown HYPOTHESIS 
 w/o sacrificing practical completeness 
 • avoid linearizations NP → PTIME 
 • forget operations PTIME → linear time + constant space

Symbolic checking FORMULA ( h ) … o 2 } ⋀ push( o 4 ,2) 
 empty 
 push(1) pop ⇒ EMPTY pop ⇒ 3 ⋀ … 
 o 6 < o 4 o 1 o 6 ⋀ pop( o 6 ,empty) 
 ⋀ … 
 } o 4 o 5 transitive 
 o 3 ⋀ o 4 < o 5 
 o 4 < o 6 pop ⇒ 1 push(2) push(3) ⋀ o 5 < o 6 THEORY (AtomicStack) ∀ o 1 ,o 2 ,o 3 . o 1 < o 2 ⋀ o 2 < o 3 ⟹ o 1 < o 3 
 transitive ∀ o 1 ,o 2 . o 1 < o 2 ⟹ o 2 ≮ o 1 
 antisymmetric ∀ o 1 ,o 2 . pop(o 1 ,empty) ⋀ push(o 2 ,_) ⋀ unmatched(o 2 ) ⟹ o 1 < o 2 
 empty … THEOREM 
 h ∈ Hist( L ) iff FORMULA ( h ) ⋀ THEORY ( L ) is SAT

Saturation Algorithm NP → PTIME ONLY PROPAGATION NO DECISIONS pop ⇒ 3 pop ⇒ 3 push(1) pop ⇒ EMP push(1) pop ⇒ EMP pop ⇒ 1 push(2) push(3) pop ⇒ 1 push(2) push(3) branch transitive backtrack pop ⇒ 3 pop ⇒ 3 pop ⇒ EMP pop ⇒ EMP empty push(2) push(3) push(2) push(3) push(2) push(2)

Operation Removal PTIME → linear time + constant space pop ⇒ 3 pop ⇒ 3 pop ⇒ EMPTY pop ⇒ EMPTY push(1) remove match m a pop ⇒ 1 push(2) push(3) push(2) push(3) t c h i n g STILL A VIOLATION THEOREM 
 Always sound to remove matches IDEA 
 identify obsolete matches

Empirically Seconds Enumerate Symbolic Symbolic+R Saturate Saturate+R 100 exponentially 75 more efficient 50 25 5 Steps 10 100 1000 E SYM SAT C(4) E SYM SAT 49 49 49 49 47 47 47 C(4) Violations discovered 44 45 40 w/o removal w/ removal SYM SAT E 35 33 33 32 complete 
 C(2) 30 27 C(4) C(2) 25 25 25 in practice E SYM SAT 20 18 18 18 C(4) 15 C(2) 15 C(2) 12 11 10 5 0 Bounded-Size k-FIFO Distributed Queue Random-Dequeue Queue Unbounded-Size k-FIFO

Conclusion monitoring scheme for checking h ∈ Hist(L) • first-order theories for describing atomic objects • incomplete solver (only boolean propagation) • inferring logical characterizations of sets of histories • Future work: Complete verification: Leverage insights on violations? • Weaker abstractions : e.g., causal consistency in place of • atomicity?

Inference Algorithm Input: a reference implementation Impl Output: FO formula representing Seq(Impl) patterns = O ; k = 1; repeat none-found = true; for each seq. hist. h of size k do if h is executable with Impl then continue; else if some pattern ⪯ h then continue; else add h to patterns; none-found = false; k++; return exclusion of patterns