Outline Parallel / Distributed Computers Air Traffic Network - PDF document

Outline ● Parallel / Distributed Computers ● Air Traffic Network Example Modeling and Simulation ● Parallel Discrete Event Simulation » Logical processes & time stamped messages » Local causality constraint and the synchronization PDES Introduction problem ● Chandy/Misra/Bryant - Null Message Algorithm The Null Message Synchronization Algorithm » Ground rules » An algorithm that doesn � t work » Deadlock avoidance using null messages 2 Maria Hybinette, UGA Maria Hybinette, UGA Parallel & Distributed Computers Shared Memory Multiprocessors . . . CPU CPU CPU ● Parallel computers (tightly coupled processors) » Shared memory multiprocessors cache cache cache Examples: » Distributed memory multicomputers Sun Enterprises ● Distributed computers (loosely coupled processors) interconnection network SGI Origin » Networked workstations . . . I/O memory memory devices Parallel Computers Distributed Computers programming model: shared variables; synchronization via locks Physical extent Machine room Building, city, global Processors Homogeneous Often heterogeneous { { shared int i; L shared int i; L Comm. Network Custom switch Commercial LAN / WAN … … Comm. Latency A few to tens of hundreds of microseconds Lock( L ) Lock( L ) Processor 1 Processor 2 (small messages) microseconds to seconds i = i + 1; i = i + 1; Unlock( L ) Unlock( L ) … … 3 4 } } Maria Hybinette, UGA Maria Hybinette, UGA Distributed Memory Multiprocessors CPU CPU Hardware Platforms memory memory cache cache Examples: . . . IBM SP Parallel Computers Distributed Computers Communications Communications Intel Paragon controllers controllers interconnection network Shared Distributed SIMD Network of Memory Memory machines Workstations (multicomputers) programming model: no shared variables: message passing Processor 1 Processor 2 { { int i; … int j; … Send( 2, &i, sizeof(int)) Receive( &j, sizeof(int)) … … } } 5 6 Maria Hybinette, UGA Maria Hybinette, UGA

Event-Oriented World View Parallel Discrete Event Simulation Event handler procedures state variables ● Extends example to model a network of airports Departure Arrival Landed Integer: InTheAir; » Encapsulate each airport simulator in a logical process Integer: OnTheGround; Event Event Event » Logical processes can schedule events (send messages) for Boolean: RunwayFree; { { { other logical processes … … … } } } More generally... Simulation application ● Physical system Simulation executive Event processing loop » Collection of interacting physical processes (airports) ● Simulation while(simulation not finished) Now = 8:45 » Collection of logical processes (LPs) E = smallest time stamp event in PEL Pending Event List (PEL) » Each LP models a physical process Remove E from PEL 9:00 10:10 » Interactions between physical processes modeled by Now := time stamp of E 9:16 scheduling events between LPs call event handler procedure 7 8 Maria Hybinette, UGA Maria Hybinette, UGA Parallel Discrete Event Simulation: Example LP Simulation Example Now: current simulation time Physical system InTheAir : number of aircraft landing or waiting to land ORD OnTheGround : number of landed aircraft RunwayFree : Boolean, true if runway available LAX JFK Arrival Event: InTheAir := InTheAir+1; physical process interactions among physical processes if( RunwayFree ) RunwayFree:=FALSE; logical process time stamped event (message) Schedule Landed event(local) @ Now + R; Simulation Landed Event: ORD InTheAir := InTheAir-1; OnTheGround := OnTheGround + 1; arrival 10:00 Schedule Departure event(local) @ Now + G; if( InTheAir > 0 ) Schedule Landed event(local) @ Now + R; else RunwayFree := True; LAX JFK Departure Event: (D = Delay to reach another airport) OnTheGround := OnTheGround - 1; all interactions between LPs must be via messages (no shared state) 9 10 Schedule Arrival Event (remote) @ (Now+D) @ another airport Maria Hybinette, UGA Maria Hybinette, UGA Parallel Discrete Event Simulation: Example The � Rub � ● LP paradigm appears well suited to concurrent execution Golden rule for each process: ● Map LPs to different processors » Multiple LPs per processor OK � Thou shalt process incoming messages in ● Communication via message passing time stamp order � » All interactions via messages local causality constraint » No shared state variables logical ORD ORD process Safe to time stamped event (message) Process? arrival arrival 10:00 10:00 SFO SFO SFO SFO 11 12 Maria Hybinette, UGA Maria Hybinette, UGA

The Synchronization Problem The Synchronization Problem Synchronization Problem: An algorithm is LPs needed to ensure each LP processes events in time stamp order ORD 10 Observation: Ignoring events with the same time stamp (for now), adherence to the local causality constraint is sufficient to ensure that the parallel simulation will produce LAX 15 20 exactly the same results as a sequential execution where all events across all LPs are processed in time stamp order. Simulation Time 13 14 Maria Hybinette, UGA Maria Hybinette, UGA Synchronization Algorithms Outline ● Conservative synchronization: Avoid violating ● Parallel / Distributed Computers the local causality constraint (wait until it’s ● Air Traffic Network Example safe to process an event) ● Parallel Discrete Event Simulation » deadlock avoidance using null messages (Chandy/ Misra/Bryant) » Logical processes » deadlock detection and recovery » Local causality constraint » synchronous algorithms (e.g., execute in � rounds � ) ● Chandy/Misra/Bryant Null Message Algorithm ● Optimistic synchronization: Allow violations » Ground rules of local causality to occur, but detect them at runtime and recover using a rollback » An algorithm that doesn � t work mechanism » Deadlock avoidance using null messages » Time Warp (Jefferson) » numerous other approaches 15 16 Maria Hybinette, UGA Maria Hybinette, UGA A Simple Conservative Algorithm Conservative Algorithms Assumptions: Algorithm A (executed by each LP): logical processes (LPs) exchanging time stamped events (messages) ● Goal: Ensure events are processed in time stamp order: static network topology, no dynamic creation of (and connection of LPs) ● messages sent on each link are sent in time stamp order while ( simulation is not over ) ● network provides reliable delivery, preserves order (received in same wait until each FIFO contains at least one message ● order that they are sent) remove smallest time stamped event from its FIFO Observation: The above assumptions imply the time stamp of the last message process that event received on a link is a lower bound on the time stamp (LBTS) of subsequent end-loop messages received on that link ORD LAX JFK ORD ● process time stamp 2 event 9 8 2 5 4 JFK ORD ● process time stamp 4 event JFK one FIFO logical ● process time stamp 5 event queue per logical process incoming link process LAX JFK ● wait ( block ) until a message is received 5 4 LAX 9 8 2 from ORD. Goal: Ensure LP processes events in time stamp order 17 18 Maria Hybinette, UGA Maria Hybinette, UGA

Deadlock Avoidance Using Null Messages Deadlock Example Break deadlock: each LP send � null � messages indicating a lower bound on the time stamp of future messages. ORD 7 (waiting 11 ORD 7 on LAX) (waiting On LAX) 8 15 JFK 10 (waiting 15 10 LAX JFK on ORD) (waiting (waiting LAX 9 8 on JFK) on ORD) (waiting 9 8 on JFK) Assume minimum delay (flight time) between airports is 3 units of time ● Recall that JFK is initially at time 5. A cycle of LPs forms where each is waiting on the next LP in the cycle. ● JFK sends null message to LAX (who is waiting for JFK) with time stamp 8 = No LP can advance; the simulation is deadlocked. (5 +3) ● LAX sends null message to ORD with time stamp 11 = (8+3) Observation : Algorithm A is prone to deadlock! (cycle of empty queues … ) 19 ● ORD may now process message with time stamp 7 Maria Hybinette, UGA Summary Deadlock Avoidance Using Null Messages Null Message Algorithm (executed by each LP): ● Parallel Discrete Event Simulation Goal : Ensure events are processed in time stamp order and avoid deadlock » Collection of sequential simulators (LPs) possibly running on different processors while( simulation is not over ) wait until each FIFO contains at least one message » Logical processes communicating exclusively by exchanging messages remove smallest time stamped event from its FIFO ● Chandy/Misra/Bryant Null Message Algorithm process that event send null messages to neighboring LPs with time stamp indicating a » Null messages: Lower bound on the time stamp of lower bound on future messages sent to that LP (current time plus future messages the LP will send lookahead ) » Null messages avoid deadlock (non-zero end-loop lookahead) The null message algorithm relies on a � lookahead � (flight time in the example) ability. 21 22 Maria Hybinette, UGA Maria Hybinette, UGA Parallel Discrete Event Simulation: Example Physical system ORD SFO JFK physical process interactions among physical processes logical process time stamped event (message) Simulation ORD arrival 10:00 SFO SFO all interactions between LPs must be via messages (no shared state) 23 Maria Hybinette, UGA