Checking Consistency Properties: Tractable Reductions to - PowerPoint PPT Presentation

Checking Consistency Properties: Tractable Reductions to Reachability Constantin Enea IRIF , University Paris Diderot

Concurrent/Distributed Objects • Modern computer software is increasingly concurrent • performance improvements often achieved by parallelizing operations • concurrent/distributed objects encapsulate concurrent shared-memory accesses into higher-level abstract data types • java.util.concurrent, Intel Thread Building Blocks • Amazon SS3, MongoDB • the widespread usage of these libraries demands for precise (formal) specifications 


Consistency Criteria • Define an abstraction of the library behaviors • Example: linearizability ensures an atomic view of the method calls deq ⇒ 1 enq(1) Thread 1 enq(2) deq ⇒ 2 Thread 2 “abstracted to” deq ⇒ 1 enq(1) Thread 1 enq(2) deq ⇒ 2 Thread 2

Modeling Library Behaviors: Histories push(1) pop ⇒ 2 Thread 1 pop ⇒ 3 pop ⇒ 1 push(2) push(3) Thread 2 happens-before partial order program order pop ⇒ 2 pop ⇒ 2 push(1) push(1) pop ⇒ 3 pop ⇒ 3 pop ⇒ 1 pop ⇒ 1 push(2) push(2) push(3) push(3)

Consistency Criteria • Expressible as (existential) second order formulas over histories • Linearizability: ( Invoc, Hb ) is linearizable i ff ( Invoc, Hb ) | = ∃ Lin. Hb ⊆ Lin ∧ Spec ( Lin ) pop ⇒ 2 push(1) pop ⇒ 3 linearized to pop ⇒ 1 push(2) push(3) push(1) pop ⇒ 1 push(2) pop ⇒ 2 push(3) pop ⇒ 3

Consistency Criteria • Expressible as second order formulas over histories • Linearizability: ( Invoc, Hb ) is linearizable i ff ( Invoc, Hb ) | = ∃ Lin. Hb ⊆ Lin ∧ Spec ( Lin ) • Sequential Consistency: ( Invoc, Po ) is sequentially consistent i ff ( Invoc, Po ) | = ∃ Lin. Po ⊆ Lin ∧ Spec ( Lin ) • Causal Consistency: …

Verifying Consistency Criteria Challenges: Checking consistency of a single history is often NP-complete • results in descriptive complexity => no first-order formulation • Checking consistency of finite-state implementations is often undecidable • Example: linearizability, sequential/causal consistency • Goal: Identifying classes of implementations/specifications for which checking • consistency is reducible to reachability (assertion checking) Polynomial procedures for checking consistency of single histories •

Results Criterion Implementations Specifications Eventual Describable using B E H, POPL’14 All Consistency "multiset automata” B E GH, POPL’17 Causal Consistency Data Independent Key-value stores Register, Stack, BE E H, ICALP’15 B E W, CONCUR’17 Linearizability Data Independent Queue, Priority Queue

Geo-Replicated Data Structures • Strong (sequential) consistency write(x,1) write(x,2)

Geo-Replicated Data Structures • Strong (sequential) consistency read(x) ⇒ 1 read(x) ⇒ 1 read(x) ⇒ 2 read(x) ⇒ 2 write(x,1) write(x,2)

Geo-Replicated Data Structures • Strong (sequential) consistency is impossible while being available and tolerating network partitions: the CAP theorem read(x) ⇒ 1 read(x) ⇒ 1 read(x) ⇒ 2 read(x) ⇒ 2 write(x,1) write(x,2)

Geo-Replicated Data Structures • Tolerating faults while preserving availability leads to anomalies w.r.t. strong (sequential) consistency read(x) ⇒ 2 read(x) ⇒ 1 read(x) ⇒ 1 read(x) ⇒ 2 write(x,1) write(x,2) Updates are seen in different orders

Verifying Causal Consistency • The set of allowed anomalies are defined by weak consistency criteria, e.g., eventual consistency, causal consistency. • Goal: Algorithmic methods for checking causal consistency. • Verifying a single history is NP-Complete • Verifying a finite-state implementation is undecidable (finite nb. of sites) Linearizability. EXPSPACE-complete Sequential Consistency. Undecidable Causal Consistency. Undecidable Eventual Consistency. Decidable

What About Key-Value Stores ? • Checking causal consistency remains undecidable in general • Restricting to data independent implementations • behaviors don’t depend on the stored/input values • For data independent implementations, it is enough to check executions where each value is written at most once

What About Key-Value Stores ? • Bad pattern = a set of operations related in a particular way • Approach: identify a set of bad patterns X such that A history is causally-consistent i ff it contains a bad pattern from X • Consequences: • checking causal consistency for a single history is polynomial time • checking causal consistency is reducible to assertion checking • define a monitor that tracks the presence of bad patterns

Definition of Causal Consistency Definitions of Causal Consistency Characterization of Causal Consistency using Bad Patterns Causal Consistency Using Bad Patterns for Verification Definition of Causal Consistency read ( x ) I 2 read ( x ) I 1 write ( x , 2) write ( x , 1) read ( x ) I 1 read ( x ) I 2 program order There exists a causality order CO such that the causal past of every read can explain its value CO includes the program (site) order 10 / 35

Definition of Causal Consistency Definitions of Causal Consistency Characterization of Causal Consistency using Bad Patterns Causal Consistency Using Bad Patterns for Verification Definition of Causal Consistency read ( x ) I 2 read ( x ) I 1 read ( x ) I 1 write ( x , 2) write ( x , 2) write ( x , 2) write ( x , 1) write ( x , 1) write ( x , 1) read ( x ) I 1 read ( x ) I 2 read ( x ) I 2 program order There exists a causality order CO such that the causal past of every read can explain its value CO includes the program (site) order 10 / 35

Causal Consistency Violations Definitions of Causal Consistency Characterization of Causal Consistency using Bad Patterns Causal Consistency Using Bad Patterns for Verification Causal Consistency Violations Causally related writes must be seen by all sites in the same order. write ( x , 1) write ( x , 2) write ( y , 3) read ( x ) I 1 11 / 35

Causal Consistency Violations Definitions of Causal Consistency Characterization of Causal Consistency using Bad Patterns Causal Consistency Using Bad Patterns for Verification Causal Consistency Violations Causally related writes must be seen by all sites in the same order. write ( x , 1) write ( x , 2) write ( y , 3) read ( x ) I 1 read ( y ) I 3 read ( x ) I 1 11 / 35

Formalizing Causal Consistency Definitions of Causal Consistency Characterization of Causal Consistency using Bad Patterns Causal Consistency Using Bad Patterns for Verification Formalizing Causal Consistency Specification = a set of sequences of operations write ( x , 1) · write ( y , 2) · read ( x ) I 1 · read ( y ) I 2 12 / 35

Formalizing Causal Consistency Definitions of Causal Consistency Characterization of Causal Consistency using Bad Patterns Causal Consistency Using Bad Patterns for Verification Formalizing Causal Consistency Specification = a set of sequences of operations write ( x , 1) · write ( y , 2) · read ( x ) I 1 · read ( y ) I 2 A history h = ( O , PO ) is causally consistent w.r.t. a specification S i ff there exists a strict partial order CO s.t. AxCausal : PO ✓ CO AxCausalValue : 8 o 2 O . CausalPast( CO , o ) v S ( CausalPast( CO , o ) = the restriction of CO to CO − 1 ( o ) [ { o } CO CausalPast ( CO, o ) o v means “can be linearized to”) 12 / 35

Di ff erentiated histories Definitions of Causal Consistency Data Independent Implementations Characterization of Causal Consistency using Bad Patterns Bad Patterns Using Bad Patterns for Verification Data Independent Implementations A history is di ff erentiated if each value is written at most once Observation : Written values do not influence behaviors. ⇒ We can assume written values are unique. write ( x , 1) write ( x , 3) write ( x , 2) write ( x , 4) read ( x ) I 3 Unicity of writes implies a canonical causality relation (included in every other causality relation). 18 / 35

Bad Patterns Definitions of Causal Consistency Data Independent Implementations Characterization of Causal Consistency using Bad Patterns Bad Patterns Using Bad Patterns for Verification Bad Patterns to Characterize Violations Bad pattern : set of operations related is a particular way Defined using the following orders: PO (program order): connects operations from the same site RF (reads-from relation): connects write to read CO (causal order): defined as ( PO ∪ RF ) + 19 / 35

Example: WriteCORead Definitions of Causal Consistency Data Independent Implementations Characterization of Causal Consistency using Bad Patterns Bad Patterns Using Bad Patterns for Verification Bad Pattern for Causal Consistency: WriteCORead Two writes w 1 and w 2 , and one read r 1 on the same variable: r 1 reads-from w 1 w 1 < CO w 2 < CO r 1 Example: write ( x , 1) write ( y , 2) write ( x , 2) read ( y ) I 2 read ( x ) I 2 read ( x ) I 1 20 / 35