D ISTRIBUTED S YSTEMS [COMP9243] D ATA VS C ONTROL R EPLICATION - PowerPoint PPT Presentation

D ISTRIBUTED S YSTEMS [COMP9243] D ATA VS C ONTROL R EPLICATION Lecture 3a: Replication & Consistency Data Replication (Server Replication/Mirroring): GNU Ftp mirror FTP FTP Server Slide 1 Slide 3 GNU Ftp mirror FTP FTP Server ➀ Replication ➁ Consistency GNU Ftp mirror FTP • Models vs Protocols ➂ Update propagation Data Replication (Caching): R EPLICATION Pop Website Cache Make copies of services on multiple machines. Why?: HTTP ➜ Reliability Web Server Pop Website • Redundancy HTTP Cache Slide 2 Slide 4 ➜ Performance • Increase processing capacity • Reduce communication HTTP ➜ Scalability (prevent centralisation) Pop Website Cache • Prevent overloading of single server ( size scalability) • Avoid communication latencies ( geographic scalability) What’s the difference between mirroring and caching? D ATA VS C ONTROL R EPLICATION 1 D ATA VS C ONTROL R EPLICATION 2

Control Replication: R EPLICATION I SSUES Slashdot HTTP Web Server Updates Apache + Perl − process request SQL − build web page ➜ Consistency (how to deal with updated data) ➜ Update propagation Slashdot HTTP Slide 5 Slide 7 Replica placement SQL Database Web Server SQL ➜ How many replicas? Apache + Perl ➜ Where to put them? − process request − build web page Slashdot Redirection/Routing HTTP ➜ Which replica should clients use? What are thethe challenges of doing this? Data and Control Replication: D ISTRIBUTED D ATA S TORE ➜ data-store stores data items Slashdot HTTP Web Server Client’s Point of View: Apache + Perl SQL − process request − build web page Client A Client B Client C Client D SQL Database Slashdot HTTP Slide 6 Slide 8 Web Server Apache + Perl SQL − process request − build web page SQL Database Slashdot HTTP We will be looking primarily at data replication Data Store (including combined data and control replication). R EPLICATION I SSUES 3 D ISTRIBUTED D ATA S TORE 4

Replica Managers: Distributed Data-Store’s Point of View: Client A Client B Client C Client D 1 issue Client A Client B Client C 4 completion Replica 1 Replica 2 Replica 3 Slide 9 Slide 11 protocol 2 Replica Replica Replica messages consistency Manager Manager Manager updates protocol Replica 1 Replica 2 Replica 3 Replica 4 3 execution Data Store Data Model: ➜ data item: simple variable ➜ data item values: explicit ( 0, 1 ), abstract ( a,b ) Timeline: ➜ data store: collection of data items ➜ ClientA/Replica1: WA(x)1 , WA(x)0 Operations on a Data Store: ➜ ClientB/Replica2: RB(x)- , RB(x)1 , RB(x)1 , RB(x)0 ➜ Read. Ri(x)b Client i performs a read for data item x and it Client A/ Slide 10 Slide 12 returns b W(x)1 W(x)0 Replica 1 ➜ Write. Wi(x)a Client i performs write on data item x setting it to a ➜ Operations not instantaneous R(x)− R(x)1 R(x)1 R(x)0 Client B/ • Time of issue (when request is sent by client) Replica 2 • Time of execution (when request is executed at a replica) • Time of completion (when reply is received by client) ➜ Coordination among replicas D ISTRIBUTED D ATA S TORE 5 C ONSISTENCY 6

C ONSISTENCY Conflicting Data: Example: ➜ Do replicas have exactly the same data? ➜ What differences are permitted? Client A: x = 1; x = 0; Possible results: - -, 11, 10, 00 Consistency Dimensions: Client B: print(x); How about 01? ➜ Time and Order print(x); What are the conflicting ops? What are the partial orders? Time: Slide 13 Slide 15 What are the total orders? ➜ How old is the data (staleness)? ➜ How old is the data allowed to be? W(x)1 W(x)0 Client A • Time, Versions R(x)0 R(x)1 Operation order: Client B ➜ Were operations performed in the right order? Can you sanely use a system like this? ➜ What orderings are allowed? Real world examples of inconsistency? O RDERING C ONSISTENCY M ODEL Updates and concurrency result in conflicting operations Defines which interleavings of operations are valid Conflicting Operations: (admissible) ➜ Read-write conflict (only 1 write) Consistency Model: ➜ Write-write conflict (multiple concurrent writes) Slide 14 Slide 16 ➜ Concerned with consistency of a data store. ➜ The order in which conflicting operations are performed affects ➜ Specifies characteristics of valid total orderings consistency A data store that implements a particular model of Partial vs Total Ordering: consistency will provide a total ordering of operations ➜ partial order: order of a single client’s operations that is valid according to the model. ➜ total order: interleaving of all conflicting operations O RDERING 7 C ONSISTENCY M ODEL 8

S TRONG O RDERING VS W EAK O RDERING Data Coherence vs Data Consistency: Strong Ordering (tight): Data Coherence ordering of operations for single data item ➜ All writes must be performed in the order that they are invoked ➜ e.g. a read of x will return the most recently written value of x ➜ Example: all replicas must see: W(x)a W(x)b W(x)c ➜ Strict (Linearisable), Sequential, Causal, FIFO (PRAM) Data Consistency ordering of operations for whole data store Slide 17 ➜ implies data coherence Slide 19 Weak Ordering (loose): ➜ includes ordering of operations on other data items too ➜ Ordering of groups of writes, rather than individual writes Non-distributed data store: ➜ Series of writes are grouped on a single replica ➜ Data coherence is respected ➜ Only results of grouped writes propagated. ➜ Program order is maintained ➜ Example: {W(x)a W(x)b W(x)c} == {W(x)a W(x)c} == {W(x)c} ➜ Weak, Release, Entry S TRICT C ONSISTENCY Any read on a data item x returns a value D ATA -C ENTRIC C ONSISTENCY M ODEL corresponding to the result of the most recent write on x A contract, between a distributed data store and Absolute time ordering of all shared accesses clients, in which the data store specifies precisely what the results of read and write operations are in W(x)a W(x)a Client A Client A Slide 18 the presence of concurrency. Slide 20 R(x)a R(x)− R(x)a Client B Client B ➜ Multiple clients accessing the same data store strictly consistent not strictly consistent ➜ Described consistency is experienced by all clients What is most recent in a distributed system? • Client A, Client B, Client C see same kinds of orderings ➜ Assumes an absolute global time ➜ Non-mobile clients (replica used doesn’t change) ➜ Assumes instant communication (atomic operation) ➜ Normal on a uniprocessor � Impossible in a distributed system S TRONG O RDERING VS W EAK O RDERING 9 L INEARISABLE C ONSISTENCY 10

C AUSAL C ONSISTENCY L INEARISABLE C ONSISTENCY Potentially causally related writes are executed in the same order everywhere All operations are performed in a single sequential order Causally Related Operations: ➜ Operations ordered according to a global (finite) timestamp. ➜ Read followed by a write (in same client) ➜ Program order of each client maintained ➜ W(x) followed by R(x) (in same or different clients) Slide 21 Slide 23 W(x)a W(x)a Client A Client A W(x)a W(x)c W(x)a W(x)c Client A Client A W(x)b W(x)b W(x)b R(x)a W(x)b Client B Client B Client B Client B R(x)a R(x)b R(x)c R(x)a R(x)b R(x)c R(x)a R(x)b R(x)b R(x)a Client C Client C Client C Client C R(x)b R(x)a R(x)c R(x)b R(x)a R(x)c Client D Client D R(x)a R(x)b R(x)b R(x)a Client D Client D causally consistent not causally consistent linearisable not linearisable How could we make this valid? S EQUENTIAL C ONSISTENCY All operations are performed in some sequential order ➜ More than one correct sequential order possible FIFO (PRAM) C ONSISTENCY ➜ All clients see the same order ➜ Program order of each client maintained Only partial orderings of writes maintained ➜ Not ordered according to time Why is this good? W(x)a W(x)c W(x)a W(x)c Client A Client A W(x)a W(x)a R(x)a W(x)b R(x)a W(x)b Slide 22 Client A Client A Slide 24 Client B Client B R(x)a R(x)b R(x)c R(x)a R(x)b R(x)c W(x)b W(x)b Client C Client C Client B Client B R(x)b R(x)a R(x)c R(x)c R(x)a R(x)b Client D Client D R(x)b R(x)a R(x)a R(x)b Client C Client C FIFO consistent not FIFO consistent R(x)b R(x)a R(x)b R(x)a Client D Client D How could we make this valid? sequential not sequential Performance: read time + write time >= minimal packet transfer time C AUSAL C ONSISTENCY 11 W EAK C ONSISTENCY 12