D ISTRIBUTED S YSTEMS [COMP9243] In a Non-Distributed System: Two - PowerPoint PPT Presentation

D ISTRIBUTED S YSTEMS [COMP9243] In a Non-Distributed System: Two approaches to communication: Lecture 4: Communication ➜ Shared memory Slide 1 Slide 3 ➀ Communication in a Distributed System ➜ Shared memory vs message passing ➜ Communication modes ➁ Communication Abstractions Shared Memory: Why Communication? x=12 i=x Cooperating processes need to communicate. Slide 2 Slide 4 x Process B Process A ➜ For synchronisation and control Shared ➜ To share data memory Address space 1 Address space 2 D ISTRIBUTED S YSTEMS [COMP9243] 1 D ISTRIBUTED S YSTEMS [COMP9243] 2

In a Non-Distributed System: In a Non-Distributed System: Two approaches to communication: Two approaches to communication: ➜ Shared memory ➜ Shared memory • Direct memory access (Threads) • Direct memory access (Threads) • Mapped memory (Processes) • Mapped memory (Processes) ➜ Message passing ➜ Message passing • OS’s IPC mechanisms Slide 5 Slide 7 Message Passing: C OMMUNICATION IN A D ISTRIBUTED S YSTEM Previous slides assumed a uniprocessor or a multiprocessor. In a distributed system (multicomputer) things change: Shared Memory: ➜ There is no way to physically share memory Slide 6 Slide 8 Message Passing: ➜ Over the network Process A Process B ➜ Introduces latencies ➜ Introduces higher chances of failure Address space 1 Address space 2 ➜ Heterogeneity introduces possible incompatibilities D ISTRIBUTED S YSTEMS [COMP9243] 3 M ESSAGE P ASSING 4

M ESSAGE P ASSING Basics: ➜ send() C OMMUNICATION M ODES ➜ receive() Data-Oriented vs Control-Oriented Communication: Variations: Data-oriented communication ➜ Connection oriented vs Connectionless ➜ Facilitates data exchange between threads ➜ Point-to-point vs Group Slide 9 Slide 11 ➜ Shared address space, shared memory & message passing ➜ Synchronous vs Asynchronous Control-oriented communication ➜ Buffered vs Unbuffered ➜ Associates a transfer of control with communication ➜ Reliable vs Unreliable ➜ Active messages, remote procedure call (RPC) & remote ➜ Message ordering guarantees method invocation (RMI) Data Representation: ➜ Marshalling ➜ Endianness Synchronous vs Asynchronous Communication: C OUPLING Synchronous Dependency between sender and receiver ➜ Sender blocks until message received • Often sender blocked until message is processed and a Temporal do sender and receiver have to be active at the reply received same time? ➜ Sender and receiver must be active at the same time Spatial do sender and receiver have to know about each ➜ Receiver waits for requests, processes them (ASAP), and returns reply other? explicitly address each other? Slide 10 Slide 12 ➜ Client-Server generally uses synchronous communication Semantic do sender and receiver have to share knowledge Asynchronous of content syntax and semantics? ➜ Sender continues execution after sending message (does not Platform do sender and receiver have to use the same block waiting for reply) platform? ➜ Message may be queued if receiver not active ➜ Message may be processed later at receiver’s convenience Tight vs Loose coupling: yes vs no When is Synchronous suitable? Asynchronous? C OMMUNICATION M ODES 5 C OMMUNICATION M ODES 6

Transient vs Persistent Communication: Direct-Addressing vs Indirect-Addressing Communication: Transient Direct-Addressing ➜ Message discarded if cannot be delivered to receiver ➜ Message sent directly to receiver immediately ➜ Example: HTTP request ➜ Example: HTTP request Slide 13 Slide 15 Indirect-Addressing Persistent ➜ Message not sent to a particular receiver ➜ Message stored (somewhere) until receiver can accept it ➜ Example: broadcast, publish/subscribe ➜ Example: email Coupling? Coupling? Combinations: A A A Provider-Initiated vs Consumer-Initiated Communication: Message can be sent only if B is Accepted running Provider-Initiated B B B ➜ Message sent when data is available Persistent Asynchronous Persistent Synchronous Transient Asynchronous ➜ Example: notifications Slide 14 Slide 16 A A A Starts processing Request Request request Accepted Accepted Consumer-Initiated ACK Received Received ➜ Request sent for data B B B Transient Synchronous Transient Synchronous Transient Synchronous ➜ Example: HTTP request (Receipt Based) (Delivery Based) (Response Based) Examples? C OMMUNICATION M ODES 7 C OMMUNICATION A BSTRACTIONS 8

C OMMUNICATION A BSTRACTIONS E XAMPLE : M ESSAGE P ASSING I NTERFACE (MPI) Abstractions above simple message passing make ➜ Designed for parallel applications communication easier for the programmer. ➜ Makes use of available underlying network Provided by higher level APIs ➜ Tailored to transient communication ➜ No persistent communication Slide 17 ➀ Message-Oriented Communication Slide 19 ➜ Primitives for all forms of transient communication ➁ Request-Reply, Remote Procedure Call (RPC) & Remote Method Invocation (RMI) ➜ Group communication ➂ Group Communication MPI is BIG. Standard reference has over 100 functions and is ➃ Event-based Communication over 350 pages long! ➄ Shared Space E XAMPLE : M ESSAGE Q UEUING S YSTEMS M ESSAGE -O RIENTED C OMMUNICATION Sender A Application Communication models based on message passing Application Receive Traditional send()/receive() provides: queue R2 Message ➜ Asynchronous and Synchronous communication Send queue Slide 18 Slide 20 ➜ Transient communication Application What more does it provide than send()/receive() ? R1 ➜ Persistent communication (Message queues) ➜ Hides implementation details Receiver B Application ➜ Marshalling Router E XAMPLE : M ESSAGE P ASSING I NTERFACE (MPI) 9 E XAMPLE : M ESSAGE Q UEUING S YSTEMS 10

Provides: ➜ Persistent communication ➜ Message Queues: store/forward E XAMPLE : R EMOTE P ROCEDURE C ALL (RPC) ➜ Transfer of messages between queues Idea: Replace I/O oriented message passing model by Model: execution of a procedure call on a remote node [BN84]: ➜ Application-specific queues ➜ Synchronous - based on blocking messages ➜ Messages addressed to specific queues Slide 21 Slide 23 ➜ Message-passing details hidden from application ➜ Only guarantee delivery to queue. Not when. ➜ Procedure call parameters used to transmit data ➜ Message transfer can be in the order of minutes ➜ Client calls local “stub” which does messaging and marshalling Examples: Confusing local and remote operations can be dangerous, ➜ IBM MQSeries, Java Message Service, Amazon SQS, Advanced why? Message Queuing Protocol, MQTT, STOMP Very similar to email but more general purpose (i.e., enables communication between applications and not just people) Remember Erlang client/server example?: % Client code using the increment server R EQUEST -R EPLY C OMMUNICATION client (Server) -> Server ! {self (), 10}, Request: receive ➜ a service {From, Reply} -> io:format ("Result: ~w~n", [Reply]) ➜ data end. Reply: Slide 22 Slide 24 % Server loop for increment server ➜ result of executing service loop () -> ➜ data receive {From, Msg} -> From ! {self (), Msg + 1}, Requirement: loop (); ➜ Message formatting stop -> true ➜ Protocol end. % Initiate the server start_server() -> spawn (fun () -> loop () end). E XAMPLE : R EMOTE P ROCEDURE C ALL (RPC) 11 E XAMPLE : R EMOTE P ROCEDURE C ALL (RPC) 12

This is what it’s like in RPC: RPC Implementation: ➀ client calls client stub (normal procedure call) % Client code client (Server) -> ➁ client stub packs parameters into message data structure register(server, Server), ➂ client stub performs send() syscall and blocks Result = inc (10), ➃ kernel transfers message to remote kernel Slide 25 Slide 27 ➄ remote kernel delivers to server stub, blocked in receive() io:format ("Result: ~w~n", [Result]). ➅ server stub unpacks message, calls server (normal proc call) % Server code ➆ server returns to stub, which packs result into message inc (Value) -> Value + 1. ➇ server stub performs send() syscall ➈ kernel delivers to client stub, which unpacks and returns Where is the communication? Example client stub in Erlang: RPC Implementation: % Client code using RPC stub Client machine Server machine client (Server) -> register(server, Server), Client process Server process Result = inc (10), Implementation 1 io:format ("Result: ~w~n", [Result]). of inc 6 Slide 26 j = inc(i); j = inc(i); Slide 28 Client stub Server stub 2 % RPC stub for the increment server proc: "inc" proc: "inc" int: val(i) int: val(i) inc (Value) -> 3 server ! {self (), inc, Value}, 5 Client OS Server OS Message receive proc: "inc" 4 {From, inc, Reply} -> Reply int: val(i) end. E XAMPLE : R EMOTE P ROCEDURE C ALL (RPC) 13 E XAMPLE : R EMOTE P ROCEDURE C ALL (RPC) 14