Cloud to the edge Just-Right Consistency Static analysis for minimal synchronisation Marc Shapiro Inria & Sorbonne-Universités UPMC-LIP6 Social, web, e-commerce: shared mutable data Scalability ⇒ replication ⇒ consistency issues [Just-Right Consistency] 2 Part I: Models matter Consistency vs. 90% Read-only transactions; Disaster Tolerant performance Termination Latency of Update Transactions (ms) Part I: Consistency vs. performance 70% Read-only transactions; Disaster Tolerant • Geo-replicated cloud databases • Consistency models 3 × • Some partial solutions Part II • Just-right consistency P-Store-SER GMU-US Serrano-SI Jessy 2pc -NMSI 4.5 × RC Walter-PSI SDUR-SER [Just-Right Consistency] [Just-Right Consistency] 3 Credit: Masoud Saeida Ardekani 4
Strongest: Weakest: Strict Serialisability Eventual consistency T1 client T1 T3 Op1 T2 Op2 R1 R1 R2 R2 R3 R3 T1 T3 T2 Invariant Invariant Invariant [Just-Right Consistency] [Just-Right Consistency] 5 6 The problem(s) of Seq. consistency consistency examples Bank account Same object: • debit(amt), credit(amt), accrueInterest(amt) • Safe: updates, state satisfy specification, internal invariants • Invariant: “balance ≥ 0” • Replicas converge to same state • { amt ≤ balance ∧ Inv } debit(amt) { Inv } Separate objects: maintain relations File system • Multi-object invariants • mkdir, rmdir, create, write, rm, ls, etc. • Different kinds ⟹ different mechanisms • Invariant: Tree ACID transactions mix all this; often too strong • { Tree ∧ ¬ x/…/y } mv(x,y) { Tree } [Just-Right Consistency] [Just-Right Consistency] 7 8
Consistency issues CAP under EC Sequential Consistency: total order of operations ⟹ replicas identical Updates delivered in different orders: not • Consensus: “Who’s next?” identical, do not converge • Requires communication Lost updates (LWW: by design) CAP Theorem: “When network can P artition, No causality: updates received out of order • either sequential C onsistency, No transactions: inter-object invariants violated • or A vailability; Compensating at application level: very • can’t have both!” challenging Availability related to performance Solution: Spanner? • Parallelise • More implementation choices [Just-Right Consistency] [Just-Right Consistency] 9 10 Operation Concurrency u u client client Convergence? u PRE u ! u ! v ! Safety? origin origin replica replica u ? u ? v ! other replica replica u ! u ! replica v ! v ! u ! u ! v ? v ? u: state ⤻ (retval, (state ⤻ state)) Concurrent, Multi-master Prepare (@origin) u ? ; deliver u ! Strong: total order, identical state Read one, write all (ROWA) Weak: concurrent, interleaving, no global state Deferred-update replication (DUR) [Just-Right Consistency] [Just-Right Consistency] 11 12
Anomalies of concurrent Not causal updates Don’t show photos to post photo Bob Bank: Alice @home u v • σ init = 100 € • Alice: credit(20) = { σ ≔ 120 } Alice @phone u v • Bob: debit (60) = { σ ≔ 40 } Bob • σ = ??? v u File system: Bob sees photo • σ init = “/“ access (Bob, photo) ⟹ ACL (Bob, photo) • Alice: mkdir (“/foo”); mkdir (“/foo/bar”) v observed effects of u • Bob: receives mkdir (“/foo/bar”) ⟹ v should be delivered after u • σ = ??? Available: doesn’t slow down sender [Just-Right Consistency] [Just-Right Consistency] 13 14 (2) Conflict-free Causal-order delivery (1) Causal consistency replicated data types Alice @home u v Data type • Encapsulates issues Alice @phone u v Replicated Bob v u • At multiple nodes Available • Update my replica without coordination access (Bob, photo) ⟹ ACL (Bob, photo) • Convergence guaranteed (formal properties) v observed effects of u • Decentralised, peer-to-peer ⟹ v should be delivered after u Available: doesn’t slow down sender [Just-Right Consistency] [Just-Right Consistency] 15 16
Commute ⟹ converge CRDT design concept Backward-compatible with sequential datatype Commute ⟹ concurrent is same Bank account: • add(e); rm(f) = rm(f); add(e) ≜ add(e) || rm (f) • credit(amt) ! = { local_balance += amt } Otherwise, concurrency semantics • debit(amt) ! = { local_balance –= amt } • Example: add(e) || rm (e) • interest() ! = • Deterministic, similar to sequential { local_balance += origin_balance* .05 } ‣ ≈ rm(e);add(e) or ≈ add(e); rm(e) File system: • Merge, don’t lose updates • write(f) ! = { local_f ⊔ f } • Result doesn't depend on order received • Stable preconditions [Just-Right Consistency] [Just-Right Consistency] 17 18 CRDT design concept CRDT design concept Backward-compatible with sequential datatype Backward-compatible with sequential datatype Commute ⟹ concurrent is same Commute ⟹ concurrent is same • add(e); rm(f) = rm(f); add(e) ≜ add(e) || rm (f) • add(e); rm(f) = rm(f); add(e) ≜ add(e) || rm (f) Otherwise, concurrency semantics Otherwise, concurrency semantics • Example: add(e) || rm (e) • Example: add(e) || rm (e) • Deterministic, similar to sequential • Deterministic, similar to sequential ‣ ≈ rm(e);add(e) or ≈ add(e); rm(e) ‣ ≈ rm(e);add(e) or ≈ add(e); rm(e) • Merge, don’t lose updates • Merge, don’t lose updates • Result doesn't depend on order received • Result doesn't depend on order received • Stable preconditions • Stable preconditions [Just-Right Consistency] [Just-Right Consistency] 19 20
CRDT concept Add-Wins Set CRDT add(1) RA {1} {1} (1, {a}, {}) (1, {a, c}, {c}) RB {1} Innovation and research: systems, algorithms, (1, {a, c}, {c}) databases add(1) remove(1) Consistency of shared mutable data RC {1} {} {1} (1, {c}, {}) (1, {c}, {c}) (1, {a, c}, {c}) [Just-Right Consistency] [CRDTs in practice] 22 21 (3) Bounded Counter CRDT types CRDT Converge concurrent updates Encapsulate replication & resolution Replicated Counter: inc(), dec() Re-usable data types Invariant: bounded “ x ≥ 0 ” Correct by construction Credit per replica: ∑ credit i ≤ bound Register Counter Asynchronous: • Last-Writer Wins • Unlimited • { credit i ≥ 1 } dec ! () ={ ctr –= 1; credit i –= 1 } • Multi-Value • Restricted ≥ 0 • transfer (credit i , credit i ) Graph Set • Directed Synchronized • Grow-Only • Monotonic DAG • 2P • acquire(credit i ) • Edit graph • Observed-Remove Sequence Map [Just-Right Consistency] [Just-Right Consistency] 23 24
SwiftCloud edge +cloud app full partial Process database database request Transmit & store T r a n DC s m update i t C C DC f a i l - o v e r Transmit Transmit C DC C Update, commit shared store locally Availability + consistency: DC switch Causal + transactional 3000+ client replicas [Just-Right Consistency] [Just-Right Consistency] 25 26 (4) NMSI: strong, parallel Antidote Non- Monotonic Wait-Free Forward Snapshot Queries Freshness y? x? x? T3 SyncFree EU project T1 x? y? T2 High performance, sharded, transactional, causal x Aims to scale to 100s of DCs • Very modular y • Partial replication T2 • Small but safe metadata (vector clock) Read from causal snapshot Mini. Commit. In DC: strong consistency, physical clocks (Clock-SI) Synch + Genuine Scalability properties: Partial Repl. Industrial apps: Virtual Wallet, SocialApp, • Wait-Free Queries configuration management, FMK • Forward Freshness • Mini. Commitment Synchronisation • Genuine Partial Replication [Just-Right Consistency] [Just-Right Consistency] 27 28
Part II: Three dimensions Just-right Gen1 / Total Linearisability Order consistency Strict Serialisability Serialisability CAP Part I: Consistency vs. performance • Geo-replicated cloud databases Snapshot Isolation • Consistency models • Some partial solutions PO / Visibility Eventual Consistency Part II Causal / • Just-right consistency Q E n o i t s i o p m o C HAT [Just-Right Consistency] [Just-Right Consistency] 29 30 Seq. consistency Application invariants examples Bank account South ⨄ Boat ⨄ North = { sheep, dog, wolf } • debit(amt), credit(amt), accrueInterest(amt) carryNorth(S) ⟹ 1 ≤ |S| ≤ 2 • Invariant: “balance ≥ 0” carrySouth(S) ⟹ 1 ≤ |S| ≤ 2 • { amt ≤ balance ∧ Inv } debit(amt) { Inv } ∀ S ∈ {South, Boat, North} : File system sheep ∈ S ∧ wolf ∈ S ⟹ dog ∈ S • mkdir, rmdir, create, write, rm, ls, etc. Hard to tease invariants out • Invariant: Tree • Silent invariants • { Tree ∧ ¬ x/…/y } mv(x,y) { Tree } [Just-Right Consistency] [Just-Right Consistency] 31 32
Recommend
More recommend
