total order broadcast VL Networked Embedded Systems Markus - PowerPoint PPT Presentation

total order broadcast VL Networked Embedded Systems Markus Kammerstetter (e0226196)

overview • basic broadcast • basic service specification and quality of service • ordering • reliability • implementation Markus Kammerstetter, 2007 2

basic broadcast (1) • different strengths of reliability (e.g. best effort – fail/recovery model) • no ordering, messages are considered „independent“ Markus Kammerstetter, 2007 3

basic broadcast (2) • two messages from same process might not be delivered in the order they were broadcast • a message m1 that causes a message m2 might be delivered by some process after m2 Markus Kammerstetter, 2007 4

basic broadcast problems • how can messages be ordered (type of ordering) ? • who orders messages ? • what degree of fault tolerance should there be ? Markus Kammerstetter, 2007 5

basic service specification (1) • broadcast service can support various properties like type of ordering or degree of fault tolerance • properties form quality of service Markus Kammerstetter, 2007 6

basic service specification (2) • interface to basic broadcast service has to be modified to support quality of service bc-send i (m, qos ): an input event of processor p i , which sends a message m to all processors, containing an indicator of the sender; qos is a parameter describing the qulity of service required Markus Kammerstetter, 2007 7

basic service specification (3) bc-recv i (m, j, qos ): an output event in which processor p i , receives the message m previously broadcast by p j; qos is a parameter describing the quality of service required Markus Kammerstetter, 2007 8

basic service specification (4) • basic broadcast service: – each bc-recv i (m, j, qos) event is mapped to an earlier bc-send i (m,qos) event – every message was previously sent (integrity) – every sent message is eventually received once (liveness) – every sent message is received only once at every processor (no duplicates) Markus Kammerstetter, 2007 9

broadcast service qualities (1) • broadcast properties can be devided in two cathegories: – ordering and reliability Markus Kammerstetter, 2007 10

broadcast service qualities (2) • ordering: – do all processors see all messages in the same order or just see the messages from a single processor in the order they were sent ? – does the order in which messages are received preserve the happens-before (causal) relation ? Markus Kammerstetter, 2007 11

broadcast service qualities (3) • reliability: – do all processors see the same set of messages even if failures occur in the underlying system ? – do all processors see all the messages broadcast by a nonfaulty processors ? Markus Kammerstetter, 2007 12

ordering (1) • single-source FIFO: for all messages m1 and m2 and all processors p i and p j , if p i sends m1 before m2, then m2 is not received at p j before m1 is. Markus Kammerstetter, 2007 13

ordering (2) • totally ordered: for all messages m1 and m2 and all processors p i and p j , if m1 is received at p i before m2 is, then m2 is not received at p j before m1 is. (all processes must deliver all messages according to the same order, i.e. the order is now total) Markus Kammerstetter, 2007 14

ordering (3) • causally ordered: for all messages m1 and m2 and every processor p i , if m1 happens before m2, then m2 is not received at p i , before m1 is. A message m1 is said to happen before m2 if either: the bc-recv event for m1 happend before the bc-send event for m2 • m1 and m2 are sent by the same processor and m1 is sent before • m2 Markus Kammerstetter, 2007 15

relationships (1) • what are the relationships between these ordering requirements ? – only one implication: causally ordered implies single- source FIFO, because the happens-before relation on messages respects the order in which individual processors send the messages – besides that implication, none of the ordering requirements implies any other Markus Kammerstetter, 2007 16

relationships (2) • causally ordered does not imply totally ordered Markus Kammerstetter, 2007 17

relationships (3) • totally ordered does not imply causally ordered or single-source FIFO ordered Markus Kammerstetter, 2007 18

relationships (4) • single-source FIFO ordered does not imply causally ordered or totally ordered Markus Kammerstetter, 2007 19

reliability (1) • as mentioned earlier, the basic broadcast service must satisfy: – integrity – liveness – no duplicates • however, in presence of faulty processors, the Liveness property needs to be weakened Markus Kammerstetter, 2007 20

reliability (2) there are at most f faulty processors and K mappings between bc-recv(m) and bc-send(m) events • integrity : for each processor p i , (0<= i <= n-1), the restriction of K to bc-recv i events is well-defined.  every received message was previously sent, no message is received „out of thin air“ Markus Kammerstetter, 2007 21

reliability (3) • no duplicates : for each processor p i , (0<= i <= n-1), the restriction of K to one-to-one.  no message is received more than once at any single processor Markus Kammerstetter, 2007 22

reliability (4) • nonfaulty liveness: when restricted to bc- send and bc-recv events at nonfaulty processors, K is surjective.  all messages broadcast by a nonfaulty processor are eventually received by all nonfaulty processors Markus Kammerstetter, 2007 23

reliability (5) • faulty liveness : if one non-faulty processor has a bc-recv event that maps to a particual bc-send event at a faulty processor, then every nonfaulty processor has such an event.  every message sent by a faulty processor is either received by all nonfaulty processors or by none of them Markus Kammerstetter, 2007 24

implementation (basic broadcast) • the basic broadcast service is using the underlying point-to-point message system with no failures. Markus Kammerstetter, 2007 25

implementation (single-source FIFO) • the single-source FIFO ordering is implemented on top of basic broadcast. • each processor assigns an incremental sequence number to each new message that it broadcasts • the recipent waits to perform the receipt of the message, until all previous messages with lower sequence numbers have been processed Markus Kammerstetter, 2007 26

implementation (single-source FIFO) N N = N + 1 bc-recv(data, N) if all previous bc-send(data, N) messages(i) (0<=i<=n) are received Markus Kammerstetter, 2007 27

implementation (total order) • total order is a more difficult property – either asymmetric: relies on central coordinator that orders all messages, (implemented on top of basic broadcast) – or symmetric: processors decide together on an order for all broadcast messages, (implemented on top of single-source FIFO) Markus Kammerstetter, 2007 28

implementation (total order) asymmetric total order algorithm: • processor p i sends m using the basic broadcast service to a unique central site at processor p c • processor p c assigns a sequence number to each message and then sends it to all processors using the basic broadcast service • a message k is received if all previous messages i (0<=i <=k) are received Markus Kammerstetter, 2007 29

implementation (total order) • to spread the communication overhead, the role of the central processor can rotate among processors (e.g. by a rotating token) • Since all messages are assigned a number in a central site, it is clear that the receives of the messages happen in the same total order at all processors Markus Kammerstetter, 2007 30

implementation (total order) symmetric total order algorithm: • based on assigning timestamps to messages • assume that underlying communication system provides single-source FIFO • each processor maintains an increasing counter (timestamp) • before sending, each message is tagged with the current timestamp Markus Kammerstetter, 2007 31

implementation (total order) • each processor also maintains a vector which estimates the timestamps of all other processors. Processor p i updates its entry for p j using the tags on messages received from p j and using special „timstamp update“ messages sent by p j . • a message with timestamp T is received if all previous messages with timestamps <=T have arrived (this is done by waiting until every vector entry is at least T) Markus Kammerstetter, 2007 32

proof (total order) proof: • we must show: – integrity – no duplicates – liveness – total ordering Markus Kammerstetter, 2007 33

proof (total order) • integrity : holds because of underlying single-source FIFO • no duplicates: holds because of underlying single-source FIFO Markus Kammerstetter, 2007 34