Just-Right Consistency Closing the CAP Gap Christopher S. Meiklejohn - PowerPoint PPT Presentation

Fælles Medicinkort • FMK [production] / FMKe [synthetic workload] 
 Danish National Joint Medicine Card; operating 24x7 since 2013 for 6 million Danish citizens • Lifecycle management for prescriptions 
 Involves patient, pharmacy, and doctor management around active prescriptions in Denmark • Assumed correct in isolation 
 “Correct-Individually”, C in ACID, each operation ensures application-level invariants 15

Fælles Medicinkort • create-prescription 
 Create prescription for patient, doctor, pharmacy • FMK [production] / FMKe [synthetic workload] 
 Danish National Joint Medicine Card; operating 24x7 • update-prescription-medication 
 since 2013 for 6 million Danish citizens Add or increase medication to prescription • Lifecycle management for prescriptions 
 Involves patient, pharmacy, and doctor management • process-prescription 
 around active prescriptions in Denmark Deliver a medication by a pharmacy • Assumed correct in isolation 
 • get-*-prescriptions 
 “Correct-Individually”, C in ACID, each operation ensures application-level invariants Query functions to return information about prescriptions 15

FMKe Invariants • Relative order [referential integrity] 
 Create a prescription and reference it by a patient 16

FMKe Invariants • Relative order [referential integrity] 
 Create a prescription and reference it by a patient • Joint update [atomicity] 
 Create prescription, then update doctor, patient, and pharmacy 16

FMKe Invariants • Relative order [referential integrity] 
 Create a prescription and reference it by a patient • Joint update [atomicity] 
 Create prescription, then update doctor, patient, and pharmacy • Precondition check [if, then] 
 Medication should not be over delivered 16

Invariants AP-compatible 17

AP-compatible • No synchronization 
 Updates occur locally without blocking, no synchronization in the critical path 18

AP-compatible • No synchronization 
 Updates occur locally without blocking, no synchronization in the critical path • Asynchronous operation 
 Updates are fast, available, and exploit concurrency 18

AP-compatible • No synchronization 
 Updates occur locally without blocking, no synchronization in the critical path • Asynchronous operation 
 Updates are fast, available, and exploit concurrency • Compatible invariants 
 Relative order and joint update invariants can be preserved 18

AP-compatibe Data Model 19

RA RB

set(1) RA 1 RB

set(1) set(2) RA 1 2 RB 3 set(3)

set(1) set(2) RA 1 2 3 RB 2 3 set(3) Concurrent assignments don’t commute!

set(1) set(2) RA 1 2 3 Assignment requires CP . RB 2 3 set(3) Concurrent assignments don’t commute!

Can we find a suitable data model for AP systems? 24

Can we find a suitable data model for AP systems? Can we make non-commutative updates commutative? 24

How do we deterministically pick a value to keep? set(1) set(2) RA 1 2 ? RB ? 3 set(3)

How do we deterministically pick a value to keep? set(1) set(2) RA 1 2 ? RB ? 3 set(3) Do we use a timestamp? (like Cassandra, and drop a value?)

Timestamps make concurrent How do we deterministically pick operations commute a value to keep? but fail to capture intent. set(1) set(2) RA 1 2 ? RB ? 3 set(3) Do we use a timestamp? (like Cassandra, and drop a value?)

Can we be smarter about the merge function? 26

Deterministic conflict resolution function. set(1) set(2) max(2,3) RA 1 2 3 max(2,3) RB 3 3 set(3)

Deterministic conflict resolution function. set(1) set(2) max(2,3) RA 1 2 3 max(2,3) RB 3 3 set(3) CRDTs generalize this framework.

Conflict-Free 
 Replicated Data Types • Replicated abstract data types 
 Extension of sequential data type that encapsulates deterministic merge function 28

Conflict-Free 
 Replicated Data Types • Replicated abstract data types 
 Extension of sequential data type that encapsulates deterministic merge function • Many existing designs 
 Sets, counters, registers, flags, maps 28

AP-compatibe Relative Order 29

RA RB

Maintain program order implication invariant. RA RB

Maintain program order implication invariant. RA RB For instance, P => Q.

true(Q) RA Q RB Make Q true.

true(Q) true(P) RA Q P RB Make P true.

Program order implies ordering relationship. true(Q) true(P) RA Q P RB

true(Q) true(P) RA Q P RB Ordering is respected at other replicas.

true(Q) true(P) RA Q P RB Out of order propagation violates invariant!

true(Q) true(P) RA Q P RB P is true, Q is NOT true!

Let’s look at a concrete example. 37

RA RB

true(Q) RA Q RB Change default administrator password.

true(Q) true(P) RA Q P RB Enable administrator login.

Replica A is secure. true(Q) true(P) RA Q P RB

true(Q) true(P) RA Q P RB Replica B is secure.

true(Q) true(P) RA Q P RB Reordering allows default password to be used to login!

Causal Consistency • Respect causality 
 Ensure updates are delivered in the causal order 
 [Lamport 78] 44

Causal Consistency • Respect causality 
 Ensure updates are delivered in the causal order 
 [Lamport 78] • Strongest available model 
 Always able to return some compatible version for an object 44

Causal Consistency • Respect causality 
 Ensure updates are delivered in the causal order 
 [Lamport 78] • Strongest available model 
 Always able to return some compatible version for an object • Referential integrity 
 Causal consistency is sufficient for providing referential integrity in an AP database 44

Causal consistency means… …relative order invariants are preserved transparently! 45

AP-compatibe Joint Update 46

RA RB C1 Client performing reads.

Create prescription. create Rx RA Rx RB C1

Add reference in doctor record. create Rx update Dr(Rx) RA Rx Dr RB C1

Add reference in patient record. create Rx update Dr(Rx) update Pt(Rx) RA Rx Dr Pt RB C1

Add reference in pharmacy record. create Rx update Dr(Rx) update Pt(Rx) update Ph(Rx) RA Rx Dr Pt Ph RB C1

Updates are causally consistent. create Rx update Dr(Rx) update Pt(Rx) update Ph(Rx) RA Rx Dr Pt Ph RB C1

create Rx update Dr(Rx) update Pt(Rx) update Ph(Rx) RA Rx Dr Pt Ph RB C1 Client can read inconsistent state.

create Rx update Dr(Rx) update Pt(Rx) update Ph(Rx) RA Rx Dr Pt Ph RB C1 Client is missing update to pharmacy.

Can we ensure updates are All-or-Nothing? 55

Group updates into an atomic transaction. create Rx update Dr(Rx) update Pt(Rx) update Ph(Rx) RA T1 RB C1

Updates reflect “All-Or-Nothing” property through snapshots. create Rx update Dr(Rx) update Pt(Rx) update Ph(Rx) RA T1 RB C1

Transactions are delivered in causal order. create Rx update Dr(Rx) update Pt(Rx) update Ph(Rx) RA T1 T2 RB C1

Therefore, snapshots are causally consistent. create Rx update Dr(Rx) update Pt(Rx) update Ph(Rx) RA T1 T2 RB C1

AP-compatible transactions provide the “A” in ACID 60

Transactional Causal Consistency Strongest model that is available (AP) 61

Invariants CAP-sensitive 62

What about preventing over delivery of prescriptions? 63

Three replicas each with two available medications. RA(2) ? RB(2) ? RC(2) ?

Replica A checks precondition and delivers medication. pp(1) RA(2) 1 1 RB(2) 1 RC(2) 1