The plan • Intro: LOCAL model, synchronicity, Bellman- Ford Network Algorithms • Subdiameter algorithms: independent sets and matchings • CONGEST model: pipelining, more matching, Boaz Patt-Shamir lower bounds Tel Aviv University 1 2 Distributed Algorithms Some Issues • Turing’s vision: multiple heads, • Different component speeds, partial failures multiple tapes, but central control • Turned out to be a major headache… • Today’s technology: hook up • … = a rich source for research components by • Higher level abstraction: Shared memory communication lines – Convenient for programmers (?) • Abstraction: – Focuses on asynchrony and failures network of processors exchanging messages Not our topic! 3 4
Our Focus: Communication The LOCAL model: Typical Tasks • Compute functions of the topology The LOCAL model • Connec�vity ≡ a graph – Spanning Trees: Breadth-First (shortest paths), • Minimum weight (MST) diameter = – Maximal Independent Set, Maximal Matching • Nodes compute, send & receive msgs • Communication tasks • No failures (nodes or links) – Broadcast, gossip, end-to-end • Running time: • In general: input/output relation – DEFINE: longest message delay ≡ one �me unit – Neglect local processing time – Dynamic version: reactive tasks 5 6 The locality of distributed problems Example: Broadcast • A source node has a message all nodes need A problem can be solved in time in LOCAL iff the following algorithm works: to learn • Each node collects full information from its – Input: environment to source at time 0 -neighborhood, and computes output • Protocol: When a new message received, send Time is the “radius” of input influence. to all neighbors ( flooding ) LOCAL time = problem locality! • Time: • Example: lower bound on time • Can be used to build a spanning tree: for approx. MIS, Matching [KMW’06] – Mark edge delivering first message as parent 7 8
Example: Find maximum Example: What’s ? • Goal: find the number of nodes in the system • Each node has an input number from • Must assume some symmetry breaking • Want to find maximum – exercise • Protocol: Whenever a new maximum • Standard assumption: Unique IDs from discovered, announce to all neighbors �(�) – Usually assume • Time to convergence: • Solution: • #messages: – Use a broadcast from each node • Converge in time • Message size: bits • #messages 9 10 Example: What’s ? Generic Algorithm Another symmetry breaking tool: randomization 1. Each node broadcasts its input Algorithm: 2. Each node computes locally its output • Time: � , #messages 1. Each node chooses a value � � 2. Find � Can do better! 3. Output ��� . 1. Build a single tree (how?) Message length: . 2. Apply algorithm in tree Can be off by a factor. Repeat and report � • #messages average to decrease variance. • Time? 11 12
Asynchrony gives trouble Asynchrony gives trouble • A tree may grow quickly in a skewed way… • A tree may grow quickly in a skewed way… …But when used for the second time, we may pay for the skew! 13 14 The Bellman-Ford Algorithm The Bellman-Ford Algorithm • Goal: Given a root, construct a shortest-paths • Goal: Given a root, construct a shortest-paths tree tree • Protocol: • Protocol: – every node – every node maintains a variable � maintains a variable � – Root has � – Root has � always always – Non-root – Non-root sets � sets � � � • Can show: stabilizes in time • Can show: stabilizes in time 15 16
B-F: Trouble With Asynchrony B-F: Trouble With Asynchrony Convergence in �(�) time, but strange things can happen… Convergence in �(�) time, but strange things can happen… 2 � 2 ��� 2 ��� 2 ��� 2 � 2 ��� 2 ��� 2 ��� 1 1 � � 0 1 2 4 0 1 2 4 0 0 0 0 0 0 0 0 0 0 • Under schedule � � : node � receives 2 � distinct messages • Schedule � � : empty schedule • Schedule � ��� : receives � messages in Allow one message: from � − � − 1 to � − � on edge of • Node 1. time units?! weight 2 � (incoming value: � ����� + 2 � ) Apply � � nodes to nodes � − �, … � 2. Allow another message: from � − � − 1 to � − � on edge of 3. weight 0 (incoming value: � ����� ) Apply � � nodes to nodes � − �, … � 4. 17 18 Synchronous model Synchronous BFS tree construction • Protocol: When first message received, mark If all processors and links run in the same speed: Execution consists of rounds . In each round: origin as parent and send to all neighbors 1. Receive messages from previous round – Input: environment to source at time 0 – Round 1: receive inputs – Break ties arbitrarily 2. Do local computation • Natural uniform “ball growing” around origin 3. Send messages • Time: Avoid skewed evolution! 19 20
Synchronizer Therefore: • Can emulate synchronous environment on • Asynchronous networks are interesting only if top of asynchronous one there may be faults • Abstraction: consistent pulse at all nodes – Or when we care about #messages How: • We henceforth assume synchronous • send a message to each synch. app. networks… pulse msgs neighbor in each round (send synch. netw. synchronizer – But we need to account for messages! null message if nothing to msgs + asynch. netw. send) • Emit pulse when received messages from all neighbors 21 22 Generic Synchronous Algorithm • Any i/o-relation solvable in diameter time: 1. Construct a BFS tree (need IDs/randomization to choose root: Leader Election!) Subdiameter algorithms 2. Send all inputs to root (“convergecast”) 3. Root computes all outputs, sends back (“broadcast”) • Ridiculous? That’s the client-server model! – Bread-and-butter distributed computing in the 70’s- 90’s, and beyond… • Interesting? Theoretically : sub-diameter upper and lower bounds 23 24
Independent Sets Independent Sets • Independent set (IS): a set of nodes, no Turan’s theorem: There always exists an IS of neighbors size . [ is the average degree] • Maximum: terribly hard Proof: By the probabilistic method . Assign labels to nodes by a random • Maximal: cannot be extended permutation. – Can be MUCH smaller than maximum IS Let be the nodes whose label is a local • Trivial sequentially! minimum . – Linear time � ��� . Lemma: • Can this parallelized? � 25 26 Lemma & Proof Distributed MIS algorithm � • Each node chooses a random label Lemma: ��� . � ��� to avoid collisions w.h.p. – Say, in Proof: By the Means Inequality. • Local minima enter MIS, neighbors eliminated � � • � • Repeat. � � �� �(�)�� � Claim: In expectation, at least half of the edges � � � � �� are eliminated in a round. � �∈� ∑ � � �� � � ��� . �∈� � 27 28
Proof of Claim Proof of Claim (cont.) � Say node is killed by neighbor if is smallest in � � ��(�) . � � � Hence: An edge is killed by � node if is killed by . � � Observation: An edge can be killed by at most � � �∈� �∈� � two nodes: the node with minimal label in and � � � �� � = 1 �(�) �(�) the node with minimal 2 � � � + �(�) + label in �(�) + �(�) �,� ∈� � � � . � 29 30 Definitions • Input: Graph G=(V,E), with weights w : E → R + • A matching: a set of disjoint edges Distributed Matching • Maximum cardinality matching (MCM) • Maximum weighted matching (MWM) 31 32
Application: Switch Scheduling Application: Switch Scheduling input output input output Goal: move packets ports ports ports ports Goal: move packets from from inputs to inputs to their outputs their outputs fabric At each time step, fabric can forward – one packet from each input – one packet to each output • To maximize throughput, find MCM! 33 34 Note: Bipartite Graphs Distributed Matching Clearly, MCM must take diameter time! V 1 V 2 • In many applications, nodes are • Information traverses the system: must decide partitioned into two subsets (input/output, boys/girls) between the following alternatives • Bipartite graphs: � = (� 1 ∪ � 2 , �) where � � ∩ � � = ∅ and � ⊆ � 1 × � 2 • Matching is simpler in this case or – Bipartite MCM: max flow – bipartite MWM: min cost flow 35 36
Recommend
More recommend
