Outline Parallel / Distributed Computers CSCI 8220 Parallel and - PDF document

Outline � � Parallel / Distributed Computers CSCI 8220 Parallel and Distributed � � Air Traffic Network Example Simulation � � Parallel Discrete Event Simulation » � Logical processes & time stamped messages » � Local causality constraint and the synchronization PDES Introduction problem � � Chandy/Misra/Bryant Null Message Algorithm The Null Message Synchronization Algorithm » � Ground rules » � An algorithm that doesn’t work » � Deadlock avoidance using null messages 2 Maria Hybinette, UGA Maria Hybinette, UGA Parallel and Distributed Computers Shared Memory Multiprocessors . . . CPU CPU CPU � � Parallel computers (tightly coupled processors) » � Shared memory multiprocessors cache cache cache Examples: » � Distributed memory multicomputers Sun Enterprises � � Distributed computers (loosely coupled processors) interconnection network SGI Origin » � Networked workstations . . . I/O memory memory devices Parallel Computers � Distributed Computers � programming model: shared variables; synchronization via locks Physical extent � Machine room � Building, city, global � Processors � Homogeneous � Often heterogeneous � { { shared int i; L shared int i; L Comm. Network � Custom switch � Commercial LAN / WAN � … … Comm. Latency � A few to tens of � hundreds of microseconds � Lock( L ) Lock( L ) Processor 1 � Processor 2 � (small messages) � microseconds � to seconds � i = i + 1; i = i + 1; Unlock( L ) Unlock( L ) … … 3 4 } } Maria Hybinette, UGA Maria Hybinette, UGA Distributed Memory Multiprocessors CPU CPU Hardware Platforms memory memory cache cache Examples: . . . IBM SP Parallel Computers Distributed Computers Communications Communications Intel Paragon controllers controllers interconnection network Shared Distributed SIMD Network of Memory Memory machines Workstations (multicomputers) programming model: no shared variables: message passing Processor 1 � Processor 2 � { { int i; … int j; … Send( 2, &i, sizeof(int)) Receive( &j, sizeof(int)) … … } } 5 6 Maria Hybinette, UGA Maria Hybinette, UGA

Event-Oriented World View Parallel Discrete Event Simulation Event handler procedures state variables � � � Extends example to model a network of airports Departure � Arrival � Landed � Integer: InTheAir; � » � Encapsulate each airport simulator in a logical process Integer: OnTheGround; � Event � Event � Event � » � Logical processes can schedule events (send messages) for Boolean: RunwayFree; � { � { � { � other logical processes … � … � … � } � More generally... } � } � Simulation application � � � Physical system Simulation executive � Event processing loop » � Collection of interacting physical processes (airports) � � Simulation while(simulation not finished) � Now = 8:45 � » � Collection of logical processes (LPs) � E = smallest time stamp event in PEL � Pending Event List (PEL) � » � Each LP models a physical process � Remove E from PEL � 9:00 � 10:10 � » � Interactions between physical processes modeled by � Now := time stamp of E � 9:16 � scheduling events between LPs � call event handler procedure � 7 8 Maria Hybinette, UGA Maria Hybinette, UGA Parallel Discrete Event Simulation: Example LP Simulation Example Now: current simulation time � Physical system InTheAir : number of aircraft landing or waiting to land ORD � OnTheGround : number of landed aircraft RunwayFree : Boolean, true if runway available SFO � JFK � Arrival Event: � InTheAir := InTheAir+1; � physical process interactions among physical processes if( RunwayFree ) � � RunwayFree:=FALSE; � logical process time stamped event (message) � Schedule Landed event(local) @ Now + R; � Simulation Landed Event: � ORD InTheAir := InTheAir-1; � OnTheGround := OnTheGround + 1; � arrival 10:00 Schedule Departure event(local) @ Now + G; � if( InTheAir > 0 ) Schedule Landed event(local) @ Now + R; � else RunwayFree := True; � SFO SFO Departure Event: (D = Delay to reach another airport) � OnTheGround := OnTheGround - 1; � all interactions between LPs must be via messages (no shared state) 9 10 Schedule Arrival Event (remote) @ (Now+D) @ another airport � Maria Hybinette, UGA Maria Hybinette, UGA Parallel Discrete Event Simulation: Example The “Rub” � � LP paradigm appears well suited to concurrent execution Golden rule for each process: � � Map LPs to different processors » � Multiple LPs per processor OK “Thou shalt process incoming messages in � � Communication via message passing time stamp order” » � All interactions via messages local causality constraint » � No shared state variables logical ORD ORD process Safe to time stamped event (message) Process? arrival arrival 10:00 10:00 SFO SFO SFO SFO 11 12 Maria Hybinette, UGA Maria Hybinette, UGA

The Synchronization Problem The Synchronization Problem Synchronization Problem: An algorithm is LPs � needed to ensure each LP processes events in time stamp order 10 � Observation: Ignoring events with the same time stamp (for now), adherence to the local causality constraint is sufficient to ensure that the parallel simulation will produce 15 � 20 � exactly the same results as a sequential execution where all events across all LPs are processed in time stamp order. Simulation Time � 13 14 Maria Hybinette, UGA Maria Hybinette, UGA Synchronization Algorithms Outline � � Conservative synchronization: avoid violating � � Parallel / Distributed Computers the local causality constraint (wait until it’s � � Air Traffic Network Example safe) � � Parallel Discrete Event Simulation » � deadlock avoidance using null messages (Chandy/ Misra/Bryant) » � Logical processes » � deadlock detection and recovery » � Local causality constraint » � synchronous algorithms (e.g., execute in “rounds”) � � Chandy/Misra/Bryant Null Message Algorithm � � Optimistic synchronization: allow violations of » � Ground rules local causality to occur, but detect them at runtime and recover using a rollback » � An algorithm that doesn’t work mechanism » � Deadlock avoidance using null messages » � Time Warp (Jefferson) » � numerous other approaches 15 16 Maria Hybinette, UGA Maria Hybinette, UGA A Simple Conservative Algorithm Conservative Algorithms Assumptions: Algorithm A (executed by each LP): logical processes (LPs) exchanging time stamped events (messages) � � Goal: Ensure events are processed in time stamp order: static network topology, no dynamic creation of LPs � � messages sent on each link are sent in time stamp order while ( simulation is not over ) � � network provides reliable delivery, preserves order wait until each FIFO contains at least one message � � remove smallest time stamped event from its FIFO Observation: The above assumptions imply the time stamp of the last message process that event received on a link is a lower bound on the time stamp (LBTS) of subsequent end-loop messages received on that link ORD SFO JFK ORD � � process time stamp 2 event 9 8 2 9 8 2 JFK � � process time stamp 4 event JFK one FIFO logical � � process time stamp 5 event queue per logical process incoming link process SFO JFK 5 4 � � wait (block ) until message is received from 5 4 SFO Goal: Ensure LP processes events in time stamp order � 17 18 Observation : Algorithm A is prone to deadlock! (cycle of empty queues…) Maria Hybinette, UGA Maria Hybinette, UGA