Global Virtual Time Wallclock time T (GVT t t ) during the execution - PDF document

Global Virtual Time Wallclock time T (GVT t t ) during the execution of a ) during the execution of a CSCI 8220 Time Warp simulation is defined as the minimum Time Warp simulation is defined as the minimum time stamp among all time stamp among all unprocessed unprocessed and and partially partially Parallel & Distributed Simulation processed messages and anti-messages in the processed messages and anti-messages in the system at wall-clock T. system at wall-clock T. PDES: Time Warp Mechanism Distributed Snapshots and Fossil Collection 2 Maria Hybinette, UGA Maria Hybinette, UGA Outline Review: Samadi’s Algorithm � Consistent Cuts � Transient message problem: » Cut points » Solution: Message acknowledgements » Cut messages � Simultaneous message problem: » Cut values » Solution: Mark acknowledgements sent after reporting local minimum � Mattern’s GVT Algorithm � Overhead: » Colors » Vector counters » Message acknowledgments: » Pipelined algorithm – Message acknowledgment for each message and anti-message. � Fossil Collection 3 4 Maria Hybinette, UGA Maria Hybinette, UGA Consistent Cuts Mattern’s Algorithm Cut point: Cut: � Asynchronous Cut message: » Executes in background concurrent with time warp Consistent cut: execution (does not require the simulation to “freeze” (i.e., block the LPs). cut � Avoids message acknowledgements LP 4 message � Approach: Based on techniques for creating LP 3 Past distributed snapshots (consistent cut) LP 2 Future LP 1 » Can some asynchronous algorithms compute exact GVT(t)? » What about synchronous algorithms? wallclock time Cut value: 5 Maria Hybinette, UGA

Consistent Cuts Consistent Cuts Cut point: an instant dividing computation into past and future Cut point: an instant dividing computation into past and future Cut: Cut: set of cut points, one per processor Cut message: Cut message: Consistent cut: Consistent cut: cut cut LP 4 LP 4 message message LP 3 LP 3 Past Past LP 2 LP 2 Future Future LP 1 LP 1 wallclock time wallclock time Cut value: Cut value: Consistent Cuts Consistent Cuts Cut point: an instant dividing computation into past and future Cut point: an instant dividing computation into past and future Cut: set of cut points, one per processor Cut: set of cut points, one per processor Cut message: a message that was sent in the past, and received in the future Cut message: a message that was sent in the past, and received in the future Consistent cut: Consistent cut: a cut where all messages crossing the cut are cut messages cut cut LP 4 LP 4 message message LP 3 LP 3 Past Past LP 2 LP 2 Future Future LP 1 LP 1 wallclock time wallclock time Cut value: Cut value: Consistent Cuts Cuts: Divides Past and Future Cut point: an instant dividing computation into past and future cut LP 4 message Cut: set of cut points, one per processor LP 3 Past Cut message: a message that was sent in the past, and received in the future LP 2 Consistent cut: a cut where all messages crossing the cut are cut messages Future LP 1 wallclock time cut LP 4 message � Consistent Cuts: Includes local state at its cut-point & all its LP 3 transient messages. Past � Observation: Time stamp of a message sent after a cut point at LP 2 wallclock time T must be at least as large as the minimum of: Future » the smallest time stamp of any unprocessed event in the processor at LP 1 time T » the smallest time stamp of any message received by the processor wallclock time after time T. Cut value: minimum among (1) local minimum of each LP at its cut point and � GVT must be smaller than or equal to both of these quantities (2) time stamp of cut messages

Observation 1 Observation 2 Asynchronous execution Execution, each LP blocks at cut point LP 4 cut T 1 > Y T 1 T 3 message LP 4 Z T 2 T 2 > min( Y, U ) LP 4 Z LP 3 Y T 3 > min( Y, U, S ) idle Past LP 3 LP 3 S Y Y Y S > X, so LP 2 X T 3 > min( Y, U, X ) U LP 2 X LP 2 X Future (Cut value � X, Y, U) LP 1 U U V V LP 1 LP 1 W W idle wallclock time Wallclock time T Any message crossing cut from future to past must have a Cut value GVT(T) time stamp > the cut value, so they can be ignored when = min( W, X, Y, Z ,U ,V ) = min ts "unprocessed message @ T computing the cut value = min( W, X, Y, Z, U, V ) Message generated by an LP after its cut point must have time � Cut value equal to GVT(T) using synchronous GVT algorithm stamp greater than the minimum of (freeze LPs: no new computations nor message sends/receives). » The LP’s local minimum at its cut point � Events generated after cut have time stamp > cut value » The time stamp of messages received after the cut point � Cut value can be used as a GVT value Mattern’s GVT Algorithm Algorithm Overview Challenge: accounting for cut messages LP 4 LP 4 LP 3 cut point LP 3 cut point LP 2 LP 2 LP 1 Wallclock time LP 1 C1 C2 Wallclock time C1 C2 Approach : � The first cut: � Construct two cuts C1, C2, approximate cut value along C2 » Changes color of each process (green to red) » Organize processes in ring, pass token around ring » Determine number of green messages sent to each process � Ensure no message that crosses C1 also cross C2 � The second cut: » Color LPs, change LP color at each cut point » Color (green/red) each message to that of LP sending message » Each process makes sure all green messages sent to it (message tag) have been received before laying down a cut point » Maintain send/receive message counters » Compute global minimum (GVT value) � GVT = min( local min along C2, time stamp of red messages ) Example: Vector Counters How does an LP know it has received all its green messages? LP 4 LP 4 LP 3 LP 3 cut point cut point LP 2 LP 2 LP 1 LP 1 Wallclock time Wallclock time C1 C2 C1 C2 � LP i maintains vector V i [1:N] , where N = #LPs Vector counters for green messages (at C2) i = j received: » V i [i] = number of green messages received by LP i V 1 V 2 V 3 V 4 » V i [r] = number of green messages sent by LP i to LP r V 1 [4] = 0 � C2: LP i cannot pass token until » V i [i] = � V s [i] (summed over all s � i ) � C1: Token includes vector to accumulate send counters

Example: Vector Counters Example: Vector Counters LP 4 LP 4 LP 3 LP 3 cut point cut point LP 2 LP 2 LP 1 LP 1 Wallclock time Wallclock time C1 C2 C1 C2 Vector counters for green messages (at C2) i = j received: Vector counters for green messages (at C2) i = j received: V 1 V 2 V 3 V 4 V 1 V 2 V 3 V 4 V 1 [4] = 0 V 1 [4] = 0 V 1 [3] = 0 V 1 [3] = 0 V 1 [2] = 1 . Example: Vector Counters Example: Vector Counters LP 4 LP 4 LP 3 LP 3 cut point cut point LP 2 LP 2 LP 1 LP 1 Wallclock time Wallclock time C1 C2 C1 C2 Vector counters for green messages (at C2) i = j received: Vector counters for green messages (at C2) i = j received: V 1 V 2 V 3 V 4 V 1 V 2 V 3 V 4 V 1 [4] = 0 V 1 [4] = 0 V 2 [4] = 0 V 1 [3] = 0 V 1 [3] = 0 V 2 [3] = 0 V 1 [2] = 1 V 1 [2] = 1 V 2 [2] = 3 V 1 [1] = -2 V 1 [1] = -2 V 2 [1] = 1 Example: Vector Counters Example: Vector Counters LP 4 LP 4 LP 3 LP 3 cut point cut point LP 2 LP 2 LP 1 LP 1 Wallclock time Wallclock time C1 C2 C1 C2 Vector counters for green messages (at C2) i = j received: Vector counters for green messages (at C2) i = j received: V 1 V 2 V 3 V 4 V 1 V 2 V 3 V 4 V 1 [4] = 0 V 2 [4] = 0 V 3 [4] = 1 V 1 [4] = 0 V 2 [4] = 0 V 3 [4] = 1 V 4 [4] = -1 V 1 [3] = 0 V 2 [3] = 0 V 3 [3] = 0 V 1 [3] = 0 V 2 [3] = 0 V 3 [3] = 0 V 4 [3] = 0 V 1 [2] = 1 V 2 [2] = 3 V 3 [2] = 0 V 1 [2] = 1 V 2 [2] = 3 V 3 [2] = 0 V 4 [2] = 2 V 1 [1] = -2 V 2 [1] = 1 V 3 [1] = 0 V 1 [1] = -2 V 2 [1] = 1 V 3 [1] = 0 V 4 [1] = 1