Write Fast, Read in the Past: Causal Consistency for Client-side - PowerPoint PPT Presentation

Challenge: Database Access for Client-side Apps Write Fast, Read in the Past: Causal Consistency for Client-side Apps with SwiftCloud App App Presented by Marek Zawirski App App Inria / UPMC-LIP6, Paris (now at Google, Zürich) Marek Zawirski, Nuno Preguiça, Sérgio Duarte, Annette Bieniusa, Valter Balegas, Marc Shapiro Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 1 Challenge: Database Access for Client-side Apps Challenge: Database Access for Client-side Apps App App App App A A A A P P P P I I I I App App App App Limited boundaries of server-side database guarantees Limited boundaries of server-side database guarantees � ad-hoc on the client-side Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 2 Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 3

Challenge: Database Access for Client-side Apps Challenge: Database Access for Client-side Apps A A A A P App P App App App P P I I I I A A A A App App P P App App P P I I I I Extended boundaries with SwiftCloud Extended boundaries with SwiftCloud Consistent, available and convergent data access • Scalability with #objects and #clients • Fault-tolerance • Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 4 Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 5 Stronger than Eventual: Causal Consistency Stronger than Eventual: Causal Consistency Default on client-side: eventual consistency � anomalies Default on client-side: eventual consistency � anomalies bob_posts.add(“don’t think of visiting Vancouver…”) bob_posts.add(“don’t think of visiting Vancouver…”) client order client order bob_posts.add(“… just do it! YOLO”) bob_posts.add(“… just do it! YOLO”) read read read replies.add(“Alice: totally J ”) replies.add(“Alice: totally J ”) Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 6 Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 7

Stronger than Eventual: Causal Consistency Stronger than Eventual: Causal Consistency Default on client-side: eventual consistency � anomalies Default on client-side: eventual consistency � anomalies bob_posts.add(“don’t think of visiting Vancouver…”) bob_posts.add(“don’t think of visiting Vancouver…”) client order client order bob_posts.add(“… just do it! YOLO”) bob_posts.add(“… just do it! YOLO”) read read read read replies.add(“Alice: totally J ”) replies.add(“Alice: totally J ”) ü !?! Causal consistency : reads from causally-closed snapshot Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 8 Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 9 Convergent Causal Consistency: No Lost Updates Convergent Causal Consistency: No Lost Updates bob_posts.add(“don’t think of visiting Vancouver…”) bob_posts.add(“don’t think of visiting Vancouver…”) client order client order bob_posts.add(“… just do it! YOLO”) bob_posts.add(“… just do it! YOLO”) read read replies.add(“Alice: totally J ”) replies.add( “Eve: Bob � ”) replies.add(“Alice: totally J ”) replies.add( “Eve: Bob � ”) replies: ??? replies: {“Alice: totally J ”, “Eve: Bob � ”} High-level convergent objects [CRDTs] resolve concurrency Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 10 Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 11

Challenge: Causal Consistency with Partial Replicas Challenge: Causal Consistency with Partial Replicas [PRACTI, NSDI’06] [PRACTI, NSDI’06] objA.op objA.op objB.op objC.op objB.op objC.op current state objA objB objB.op objB.op Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 12 Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 13 Challenge: Causal Consistency with Partial Replicas Challenge: Causal Consistency with Partial Replicas [PRACTI, NSDI’06] [PRACTI, NSDI’06] objA.op objA.op objB.op objC.op objB.op objC.op objA objA objB objB objB.op objB.op objA objB objB objC objC objC Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 14 Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 15

Challenge: Causal Consistency with Partial Replicas Challenge: Causal Consistency with Partial Replicas [PRACTI, NSDI’06] [PRACTI, NSDI’06] objA.op objA.op objB.op objC.op objB.op objC.op objA objA objB objB objB.op objB.op objA objA objB.op objB objB objC objC dependencies objC objC objA.op objB.op objC.op Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 16 Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 17 Challenge: Causal Consistency with Partial Replicas Approach: Cloud-backed Partial Replicas [PRACTI, NSDI’06] Data Center full replicas: objA.op ü Provide consistent view Assign small metadata ü 4 9 objB.op objC.op geo-replication DC DC objA objB objB.op objA objB.op objB objC dependencies objC Inherent trade-offs in the general case : 1 0 2 objB.op scalability vs. availability vs. fault-tolerance Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 18 Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 19

Approach: Cloud-backed Partial Replicas Approach: Cloud-backed Partial Replicas Data Center full replicas: Data Center full replicas: ü Provide consistent view Assign small metadata ü Provide consistent view Assign small metadata ü ü 4 9 4 9 geo-replication geo-replication DC DC DC DC log log cache cache App App update Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 20 Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 21 Approach: Cloud-backed Partial Replicas Approach: Cloud-backed Partial Replicas Data Center full replicas: Data Center full replicas: ü Provide consistent view Assign small metadata ü Provide consistent view Assign small metadata ü ü 4 9 4 9 geo-replication geo-replication DC DC DC DC 4 9 5 cache log cache log App App update read Client reads: cached fragment of cloud version � own log ü ü High availability Consistency w/read-your-writes Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 22 Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 23

Potential of Cloud-backed Client Replicas Potential of Cloud-backed Client Replicas CDF for 1000 clients CDF for 1000 clients Response time of operation [ms] Response time of operation [ms] Objects in the cache � immediate, consistent response Setup: DCs in 3 AWS EC2 regions, YCSB workload, cache=256 objects Setup: DCs in 3 AWS EC2 regions, YCSB workload, cache=256 objects Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 24 Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 25 Challenge for the Cloud Approach: Safe DC Failover Challenge for the Cloud Approach: Safe DC Failover log log cache cache risky read Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 26 Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 27

Challenge for the Cloud Approach: Safe DC Failover Challenge for the Cloud Approach: Safe DC Failover operations with operations with risky dependencies risky dependencies log log cache cache risky read risky read new DC in incompatible state Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 28 Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 29 Supporting Failover by Conservative Reads Supporting Failover by Conservative Reads log log cache cache K =2 K =2 conservative read Foreign updates : read version replicated in K > 1 DCs Foreign updates : read version replicated in K > 1 DCs Own writes : read from the log, recover to a new DC Own writes : read from the log, recover to a new DC Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 30 Zawirski et al., Write Fast, Read in the Past: Causal Consistency for Client-side Applications with SwiftCloud 31