Sorting and Other Distributed Set Operations T-79.4001 Seminar on - PowerPoint PPT Presentation

Sorting a Distributed Set Distributed Set Operations Sorting and Other Distributed Set Operations T-79.4001 Seminar on Theoretical Computer Science Spring 2007 – Distributed Computation Eero Häkkinen 2007-04-18 Based on sections 5.3-5.4 of N. Santoro: Design and Analysis of Distributed Algorithms, Wiley 2007. Eero Häkkinen Sorting and Other Distributed Set Operations

Sorting a Distributed Set Distributed Set Operations Sorting a Distributed Set 1 Introduction OddEven-LineSort OddEven-MergeSort Lower Bounds SelectSort DynamicSelectSort Distributed Set Operations 2 Introduction Intersection Difference Partitioning (IDP) Local Evaluation Global Evaluation Eero Häkkinen Sorting and Other Distributed Set Operations

Introduction OddEven-LineSort Sorting a Distributed Set OddEven-MergeSort Distributed Set Operations Lower Bounds SelectSort DynamicSelectSort Definitions (1/3) Notation (1/2) a local set D x in an entity x � a distributed set D = D x x a distribution D = � D x 1 , D x 2 , . . . , D x n � of D among the entities x 1 , x 2 , . . . , x n � the number of data items N = | D x | x a topology G of the network Eero Häkkinen Sorting and Other Distributed Set Operations

Introduction OddEven-LineSort Sorting a Distributed Set OddEven-MergeSort Distributed Set Operations Lower Bounds SelectSort DynamicSelectSort Definitions (2/3) Notation (2/2) a permutation π of the indices { 1 , 2 , . . . , n } an i th item π ( i ) of π (if π = � 2 , 4 , 1 , 3 � , then π ( 2 ) = 4) For Simplicity id ( x i ) = i D i denotes D x i Eero Häkkinen Sorting and Other Distributed Set Operations

Introduction OddEven-LineSort Sorting a Distributed Set OddEven-MergeSort Distributed Set Operations Lower Bounds SelectSort DynamicSelectSort Definitions (3/3) Sorting Condition The distribution � D 1 , D 2 , . . . , D n � is sorted according to π if and only if the following sorting condition holds: i < j ⇒ ∀ d ′ ∈ D π ( i ) , d ′′ ∈ D π ( j ) : d ′ < d ′′ Some Sorting Orders increasing order : π = � 1 , 2 , . . . , n � decreasing order : π = � n , ( n − 1 ) , . . . , 1 � Eero Häkkinen Sorting and Other Distributed Set Operations

Introduction OddEven-LineSort Sorting a Distributed Set OddEven-MergeSort Distributed Set Operations Lower Bounds SelectSort DynamicSelectSort Sorting Problem Sorting Problem Then the initial distribution of D is D = � D 1 , D 2 , . . . , D n � , the problem is to move data items among the entities so that the final distribution of D is D ′ = � � D ′ 1 , D ′ 2 , . . . , D ′ and the n distribution D ′ is sorted according to π . Notes No relation is defined between D i and D ′ i , yet. There are thus multiple variations of the problem. Eero Häkkinen Sorting and Other Distributed Set Operations

Introduction OddEven-LineSort Sorting a Distributed Set OddEven-MergeSort Distributed Set Operations Lower Bounds SelectSort DynamicSelectSort Sorting and Distribution Types Fundamental Requirements invariant-sized sorting: � � � = | D i | , 1 ≤ i ≤ n � D ′ i equidistributed sorting: � � N � � � if 1 ≤ i < n , � D ′ � � n � = � N � π ( i ) N − ( n − 1 ) if i = n , n compacted sorting:  if 1 ≤ i < N w , � �  w � N � � D ′ � � � = N − ( i − 1 ) w , if i = ≤ n , where π ( i ) w � N �  0 , if < i ≤ n w � N � w ≥ is the storage capacity of the entities n Eero Häkkinen Sorting and Other Distributed Set Operations

Introduction OddEven-LineSort Sorting a Distributed Set OddEven-MergeSort Distributed Set Operations Lower Bounds SelectSort DynamicSelectSort Description of OddEven-LineSort (1/2) Restrictions Standard restrictions R . � � , 1 ≤ i < n . Ordered line: links x π ( i ) , x π ( i + i ) Origin Based on the parallel algorithm odd-even-transposition sort, which is based on the serial algorithm bubble sort. Eero Häkkinen Sorting and Other Distributed Set Operations

Introduction OddEven-LineSort Sorting a Distributed Set OddEven-MergeSort Distributed Set Operations Lower Bounds SelectSort DynamicSelectSort Description of OddEven-LineSort (2/2) Technique In an odd iteration, entities x π ( 2 i + 1 ) and x π ( 2 i + 2 ) , 1 � n � 0 ≤ i ≤ − 1 exchange data items. The smallest items 2 are retained by x π ( 2 i + 1 ) and the largest ones are retained by x π ( 2 i + 2 ) . In an even iteration, entities x π ( 2 i ) and x π ( 2 i + 1 ) , 2 � n � 1 ≤ i ≤ − 1 exchange data items. The smallest items 2 are retained by x π ( 2 i ) and the largest ones are retained by x π ( 2 i + 1 ) . If no items change the place in an iteration other than the 3 first one, the process stops. Eero Häkkinen Sorting and Other Distributed Set Operations

Introduction OddEven-LineSort Sorting a Distributed Set OddEven-MergeSort Distributed Set Operations Lower Bounds SelectSort DynamicSelectSort Properties of OddEven-LineSort Complexity Sorting an equidistributed distribution requires at most n − 1 iterations if the required sorting is invariant-sized, equidistributed or compacted. Invariant-sized sorting requires at most N − 1 iterations. T [ OddEven − LineSort invariant ] = O ( nN ) M [ OddEven − LineSort invariant ] = O ( nN ) Eero Häkkinen Sorting and Other Distributed Set Operations

Introduction OddEven-LineSort Sorting a Distributed Set OddEven-MergeSort Distributed Set Operations Lower Bounds SelectSort DynamicSelectSort Description of OddEven-Merge (1/2) Restrictions Standard restrictions R . Complete graph. Initially � � Sorted partial distributions A 1 , A 2 , . . . , D p and 2 � � A p 2 + 1 , A p 2 + 2 , . . . A p . For simplicity, p is a power of 2. Eero Häkkinen Sorting and Other Distributed Set Operations

Introduction OddEven-LineSort Sorting a Distributed Set OddEven-MergeSort Distributed Set Operations Lower Bounds SelectSort DynamicSelectSort Description of OddEven-Merge (2/2) Technique If p = 2, there are two entities y 1 and y 2 containing sets A 1 1 and A 2 . Entities y 1 and y 2 exchange data items. The smallest items are retained by y 1 and the largest ones are retained by y 2 . This is a merge . If p > 2, 2 Recursively OddEven-Merge partial distributions 1 � � � � A 1 , A 3 , . . . , A p and A p 2 + 1 , A p 2 + 3 , . . . , A p − 1 . 2 − 1 Recursively OddEven-Merge partial distributions 2 � � � � A 2 , A 4 , . . . , A p and A p 2 + 2 , A p 2 + 4 , . . . , A p . 2 Merge A 2 i and A 2 i + 1 , 1 ≤ i ≤ p 2 − 1. 3 Eero Häkkinen Sorting and Other Distributed Set Operations

Introduction OddEven-LineSort Sorting a Distributed Set OddEven-MergeSort Distributed Set Operations Lower Bounds SelectSort DynamicSelectSort Description of OddEven-MergeSort Technique Recursively OddEven-MergeSort the partial distribution 1 � � D 1 , D 2 , . . . , D n . 2 Recursively OddEven-MergeSort the partial distribution 2 � � D n 2 + 1 , D n 2 + 2 , . . . , D n . � � OddEven-Merge partial distributions D 1 , D 2 , . . . , D n and 3 2 � � D n 2 + 1 , D n 2 + 2 , . . . , D n . Eero Häkkinen Sorting and Other Distributed Set Operations

Introduction OddEven-LineSort Sorting a Distributed Set OddEven-MergeSort Distributed Set Operations Lower Bounds SelectSort DynamicSelectSort Properties of OddEven-MergeSort Complexity Sorting requires at most 1 + log n iterations. M [ OddEven − MergeSort ] = O ( N log n ) Correctness Does it work? Not always. Eero Häkkinen Sorting and Other Distributed Set Operations

Introduction OddEven-LineSort Sorting a Distributed Set OddEven-MergeSort Distributed Set Operations Lower Bounds SelectSort DynamicSelectSort Lower Bounds – Analysis Sorting Problem (recapitulation) Then the initial distribution of D is D = � D 1 , D 2 , . . . , D n � , the problem is to move data items among the entities so that the � � final distribution of D is D ′ = D ′ 1 , D ′ 2 , . . . , D ′ and the n distribution D ′ is sorted according to π . Messages � � � � D i ∩ D ′ � � items to be moved from x i to x j . j � � At least d G x i , x j messages for each item to be moved. � � � � � � D i ∩ D ′ � � The total cost at least C ( D , G , π ) = � d G x i , x j . j i � = j Eero Häkkinen Sorting and Other Distributed Set Operations

Introduction OddEven-LineSort Sorting a Distributed Set OddEven-MergeSort Distributed Set Operations Lower Bounds SelectSort DynamicSelectSort Lower Bounds – Values Ordered Line � � � � � � D i ∩ D ′ � � C ( D , G , π ) = � d G x i , x j = Ω ( nN ) j i � = j OddEven-LineSort has O ( nN ) . The same! Complete Graph � � � � � � D i ∩ D ′ � � C ( D , G , π ) = � d G x i , x j = Ω ( N ) j � �� � i � = j = 1 OddEven-MergeSort has O ( N log n ) . Eero Häkkinen Sorting and Other Distributed Set Operations