Communication Middleware Software Layers 1.1 A distributed system - PowerPoint PPT Presentation

Communication Middleware

Software Layers 1.1 A distributed system organized as middleware. Note that the middleware layer extends over multiple machines. 2

Communication Alternatives ฀ Raw message passing ฀ TCP/IP: reliable byte stream ฀ UDP/IP: unreliable, packet oriented, connectionless Problem: low level --> not easy to use ฀ OS Abstractions: Distributed Shared Memory Approach: on page fault, get the remote page Pros: easy to program. Cons: high overhead, hard to handle failures 3

RPC (Xerox RPC) n Remote Procedure Call (RPC) ฀ Main idea: Call remote procedures as if they are local ones. ฀ Key adv.: general, easy to use, and efficient. Uses the well-known procedure call semantics. l The caller makes a procedure call and then waits. If it is a local l procedure call, then it is handled normally; if it is a remote procedure, then it is handled as a remote procedure call. Caller semantics is blocked send; callee semantics is blocked receive to l get the parameters and a nonblocked send at the end to transmit results. 4

RPC Architecture 5

Example Interface{ // has enough information for compile time // checking, and generating proper calling sequence int func1(int arg) int func2(char * buffer, int size) } 6

Example: Client stub int func1(int arg) Client stub for func1 int func1(int arg) { Create req Pack table_index, fid, arg, …etc Send req Wait for reply Unpack results Return result } 7

Example: Server stub int func1(int arg) Server stub fun1{ Unpack request Find the requested function Call the function Pack results Send results } 8

Example: How about fun2 Interface{ // has enough information for compile time // checking, and generate proper calling sequence int func1(int arg) int func2(char * buffer, int size) } Problems: ฀ *buffer points into different address space. (solution: copy/restore if needed) ฀ Can be input or output (solution: add an argument type (in|out|both) ฀ Complex data structures 9

Binding Example RPC-based file system Server function id (fid), unique id that is generated Load function in memory every time the function is exported Populate export table (fid, fname, call_back function) Export interface Registry Store s_ip, fname 10

Binding Example: RPC-based file system Client Query registry Registry Query server Server Export table Store info in Index Proc. fid call back Memory (fname, server_ip, fid, index) Name pointer 11

Runtime Export table Index Proc. fid call back Client Name pointer call fun1() Server Check export table Run function Return result Runtime choices: TCP/IP? - High overhead on server, many connections and state - High latency (setting/closing connections) UDP/IP - Unreliable. (used by Xerox RPC) 12

Fault tolerance Lost request Client call fun1() Server time out Run function 13

Fault tolerance Lost reply Client call fun1() Server time out Run function Run function Problem? function executed twice!! 14

Call semantics ฀ At least once: the function is executed at least once. ฀ At most once: the function is executed once or not at all Neither one is perfect: exactly-once is desired but not possible with a single server RPC. Xerox RPC implements, at most once semantic: ฀ If success is returned  exactly once: it is guaranteed that it was executed exactly once ฀ On failure: at most once. 15

Xerox at most once semantics Unique Export table To avoid duplicates processes id Strictly increasing Index Proc. fid call back sequence number Name pointer Client call fun1() Server Check export table Check prev. calls table Run function? (client ip, process ID, seq #, result) Return result client_ip Proc. Id Seq# 16

Fault tolerance Lost reply Client client_ip Proc. Id Seq# call fun1() IP1 22 10 Server time out Update the prev. call table Run function Check prev. calls tbl Duplicate, return without execution 17

Fault tolerance Client failure with repeated seq#? Client client_ip Proc. Id Seq# call fun1() IP1 22 10 Server Run function reboot Check prev. calls tbl Later call of ->Duplicate, return fun1() Solution: add clock based conversation id, to Fun1() was acked a second time every call. without execution. <Conversation_id, seq#> is strictly increasing 18

Fault tolerance: Server crash Client call fun1() client_ip Proc. Id Seq# IP1 22 10 Server time out Run function update prev. calls tbl reboot Run function again !! Solution: Rebind on server restart with new fid 19

Fault tolerance: Server crash Client call fun1() Server Run function time out reboot Rebind functions with a different fid fid does not match What should the application do with this error? 20

Message-Based Communication ฀ Lower-level interface to provide more flexibility ฀ Popular systems (RabbitMQ, ActiveMQ, Kafka) ฀ Two (abstract) primitives are used to implement these ฀ send ฀ receive ฀ Issues: ฀ Are primitives blocking or nonblocking (synchronous or asynchronous)? ฀ Are primitives reliable or unreliable (persistent or transient)? 3-21 CS454/654

Message-Queuing System: P2P ฀ The relationship between queue-level addressing and network-level addressing. 3-22 CS454/654 From Tanenbaum and van Steen, Distributed Systems: Principles and Paradigms

Message Brokers ฀ The general organization of a message broker in a message-queuing system. 2-30 3-23 CS454/654 From Tanenbaum and van Steen, Distributed Systems: Principles and Paradigms

Message-Queuing System: flexible MQ network 2-29 3-24 CS454/654 From Tanenbaum and van Steen, Distributed Systems: Principles and Paradigms

Synchronous/Asynchronous Messaging ฀ Synchronous ฀ The sender is blocked until its message is stored in the local buffer at the receiving host or delivered to the receiver. ฀ Asynchronous ฀ The sender continues immediately after executing a send ฀ The message is stored in the local buffer at the sending host or at the first communication server. 3-25 CS454/654

Transient/Persistent Messaging ฀ Transient ฀ The sender puts the message on the net and if it cannot be delivered to the sender or to the next communication host, it is lost. ฀ There can be different types depending on whether it is asynchronous or synchronous ฀ Persistent ฀ The message is stored in the communication system as long as it takes to deliver the message to the receiver 3-26 CS454/654