Global State and Gossip CS 240: Computing Systems and Concurrency Lecture 6 Marco Canini Credits: Indranil Gupta developed much of the original material.
Today 1. Global snapshot of a distributed system 2. Chandy-Lamport’s algorithm 3. Gossip 2
Distributed snapshot • Let’s think of this as a picture of all servers and their states comprising a distributed system • How do you calculate a “global snapshot” in a distributed system? • What does a “global snapshot” even mean? • Why is the ability to obtain a “global snapshot” important? 3
Some uses of global system snapshot • Checkpointing – can restart distributed system on failure • Gargabe collection of objects – objects at servers that don’t have any other objects (at any servers) with references to them • Deadlock detection – useful in database transaction systems • Termination of computation – useful in batch computing systems • Debugging – useful to inspect the global state of the system 4
What’s a global snapshot? • Global Snapshot = Global State = Individual state of each process in the distributed system + Individual state of each communication channel in the distributed system • Capture the instantaneous state of each process • And the instantaneous state of each communication channel, i.e., messages in transit on the channels 5
A strawman solution • Synchronize clocks of all processes • Ask all processes to record their states at known time t • Problems? – Time synchronization always has error • Your bank might inform you, “We lost the state of our distributed cluster due to a 1 ms clock skew in our snapshot algorithm.” – Also, does not record the state of messages in the channels • Again: synchronization not required – causality is enough! 6
Example P i C ij C ji P j 7
[$1000, P i 100 iPhones] C ij [empty] [empty] C ji P j [$600, 50 Androids] [Global Snapshot 0] 8
[$701, P i 100 iPhones] C ij [empty] [$299, Order Android ] C ji P j [$600, 50 Androids] [Global Snapshot 1] 9
[$701, P i 100 iPhones] C ij [$499, Order [$299, Order Android ] iPhone] C ji P j [$101, 50 Androids] [Global Snapshot 2] 10
[$1200, 1 iPhone order from P j , P i 100 iPhones] C ij [empty] [$299, Order Android ] C ji P j [$101, 50 Androids] [Global Snapshot 3] 11
[$1200, P i 99 iPhones] C ij [empty] [ ($299, Order Android), (1 iPhone) C ji ] P j [$101, 50 Androids] [Global Snapshot 4] 12
[$1200, P i 99 iPhones] C ij [empty] [ (1 iPhone) ] C ji P j [$400, 1 Android order from P i , 50 Androids] [Global Snapshot 5] 13
[$1200, P i 99 iPhones] C ij [empty] [empty] … and so on … C ji P j [$400, 1 Android order from P i , 50 Androids, 1 iPhone] [Global Snapshot 6] 14
Moving from State to State • Whenever an event happens anywhere in the system, the global state changes –Process receives message –Process sends message –Process takes a step • State to state movement obeys causality –Next: Causal algorithm for Global Snapshot calculation 15
Today 1. Global snapshot of a distributed system 2. Chandy-Lamport’s algorithm 3. Gossip 16
System Model • Problem: Record a global snapshot (state for each process, and state for each channel) • System Model: – N processes in the system – There are two uni-directional communication channels between each ordered process pair P j à P i and P i à P j – Communication channels are FIFO-ordered • First in First out – No failure – All messages arrive intact, and are not duplicated • Other papers later relaxed some of these assumptions 17
Requirements • Snapshot should not interfere with normal application actions, and it should not require application to stop sending messages • Each process is able to record its own state – Process state: Application-defined state or, in the worst case: – its heap, registers, program counter, code, etc. (essentially the coredump) • Global state is collected in a distributed manner • Any process may initiate the snapshot – We’ll assume just one snapshot run for now 18
Chandy-Lamport Global Snapshot Algorithm • First: Initiator P i records its own state • Initiator process creates special messages called “Marker” messages – Not an application message, does not interfere with application messages • for j=1 to N except i • P i sends out a Marker message on outgoing channel C ij • ( N-1 ) channels • Starts recording the incoming messages on each of the incoming channels at P i : C ji (for j=1 to N except i ) 19
Chandy-Lamport Global Snapshot Algorithm (2) Whenever a process P i receives a Marker message on an incoming channel C ki • if (this is the first Marker P i is seeing) – P i records its own state first – Marks the state of channel C ki as “empty” – for j=1 to N except i • P i sends out a Marker message on outgoing channel C ij – Starts recording the incoming messages on each of the incoming channels at P i : C ji (for j=1 to N except i and k ) • else // already seen a Marker message – Mark the state of channel C ki as all the messages that have arrived on it since recording was turned on for C ki 20
Chandy-Lamport Global Snapshot Algorithm (3) The algorithm terminates when • All processes have received a Marker – To record their own state • All processes have received a Marker on all the ( N-1 ) incoming channels at each – To record the state of all channels Then, (if needed), a central server collects all these partial state pieces to obtain the full global snapshot 21
Example A B C D E P1 Time E F G P2 H I J P3 Instruction or Step Message 22
P1 is Initiator: • Record local state S1, • Send out markers • Turn on recording on channels C 21 , C 31 A B C D E P1 Time E F G P2 H I J P3 23
S1, Record C 21 , C 31 A B C D E P1 Time E F G P2 H I J P3 • First Marker! • Record own state as S3 • Mark C 13 state as empty • Turn on recording on other incoming C 23 • Send out Markers 24
S1, Record C 21 , C 31 A B C D E P1 Time E F G P2 H I J P3 • S3 • C 13 = < > • Record C 23 25
Duplicate Marker! State of channel C 31 = < > S1, Record C 21 , C 31 A B C D E P1 Time E F G P2 H I J P3 • S3 • C 13 = < > • Record C 23 26
C 31 = < > S1, Record C 21 , C 31 A B C D E P1 Time E F G P2 H I J P3 • First Marker! • S3 • Record own state as S2 • C 13 = < > • Mark C 32 state as empty • Turn on recording on C 12 • Record C 23 • Send out Markers 27
C 31 = < > S1, Record C 21 , C 31 A B C D E P1 Time E F G P2 H I J P3 • S3 • S2 • C 13 = < > • C 32 = < > • Record C 23 • Record C 12 28
C 31 = < > S1, Record C 21 , C 31 A B C D E P1 Time E F G P2 H I J P3 • S3 • S2 • Duplicate! • C 12 = < > • C 13 = < > • C 32 = < > • Record C 23 • Record C 12 29
• Duplicate! • C 21 = <message G à D > C 31 = < > S1, Record C 21 , C 31 A B C D E P1 Time E F G P2 H I J P3 • S3 • S2 C 12 = < > • C 13 = < > • C 32 = < > • Record C 23 • Record C 12 30
• C 21 = <message G à D > C 31 = < > S1, Record C 21 , C 31 A B C D E P1 Time E F G P2 H I J P3 • S3 • S2 C 12 = < > • C 13 = < > • C 32 = < > • Record C 23 • Record C 12 • Duplicate! • C 23 = < > 31
Recommend
More recommend
