Mapping filter services on heterogeneous platforms To appear in - PowerPoint PPT Presentation

Mapping filter services on heterogeneous platforms To appear in IPDPS 2009 Anne Benoit,Fanny Dufoss´ e,Yves Robert January 9, 2009 Anne Benoit,Fanny Dufoss´ e,Yves Robert () Mapping filter services on heterogeneous platforms January 9, 2009 1 / 42

Introduction The problem: treatment of a data flow filter services with selectivity σ and cost c precedence constraints between services servers with speed s one-to-one mappings The objective: minimize the period minimize the latency Anne Benoit,Fanny Dufoss´ e,Yves Robert () Mapping filter services on heterogeneous platforms January 9, 2009 2 / 42

Motivation For services of selectivity less than one grep web services Select-Project-Join query optimization ... Related problems: component testing unsupervised systems Anne Benoit,Fanny Dufoss´ e,Yves Robert () Mapping filter services on heterogeneous platforms January 9, 2009 3 / 42

Framework 1 Period 2 General structure of optimal solutions Case of homogeneous servers NP-completeness of MinPeriod - Het Integer linear program Latency 3 General structure of optimal solutions Polynomial algorithm on homogeneous platforms NP-completeness of problem MinLatency - NoPrec - Het Integer linear program Bi-criteria problem 4 Heuristics 5 Experiments 6 Conclusion 7 Anne Benoit,Fanny Dufoss´ e,Yves Robert () Mapping filter services on heterogeneous platforms January 9, 2009 4 / 42

The instances The problems depend on: the criteria: MinPeriod , MinLatency or BiCriteria the platform: Hom or Het the dependence constraints: NoPrec or Prec Anne Benoit,Fanny Dufoss´ e,Yves Robert () Mapping filter services on heterogeneous platforms January 9, 2009 5 / 42

The instances The problems depend on: the criteria: MinPeriod , MinLatency or BiCriteria the platform: Hom or Het the dependence constraints: NoPrec or Prec The instances: A = ( F , G , S ) with: The services: F = { C 1 , C 2 , . . . , C n } The precedence constraints: G ⊂ F × F The servers: S = { S 1 , S 2 , . . . , S p } Anne Benoit,Fanny Dufoss´ e,Yves Robert () Mapping filter services on heterogeneous platforms January 9, 2009 5 / 42

The problem Example for 3 independent services: The plan? C 1 C 1 C 2 C 3 C 3 C 2 C 1 C 3 C 2 C 2 C 1 C 3 The mapping? ( C 1 , S 2 ) , ( C 2 , S 1 ) , ( C 3 , S 3 ) ( C 1 , S 3 ) , ( C 2 , S 2 ) , ( C 3 , S 1 ) Anne Benoit,Fanny Dufoss´ e,Yves Robert () Mapping filter services on heterogeneous platforms January 9, 2009 6 / 42

Example C 1 C 1 C 2 C 3 C 3 C 2 Figure: Chaining services. Figure: Combining selectivities � � � � c 1 s 1 , σ 1 c 2 s 2 , σ 1 σ 2 c 3 c 1 s 1 , c 2 s 2 , σ 1 σ 2 c 3 P = max P = max s 3 s 3 � � L = c 1 s 1 + σ 1 c 2 s 2 + σ 1 σ 2 c 3 c 1 s 1 , c 2 + σ 1 σ 2 c 3 L = max s 3 s 2 s 3 Anne Benoit,Fanny Dufoss´ e,Yves Robert () Mapping filter services on heterogeneous platforms January 9, 2009 7 / 42

Example c 1 = 1, c 2 = 4, c 3 = 10 σ 1 = 1 2 , σ 2 = σ 3 = 1 3 s 1 = 1, s 2 = 2 and s 3 = 3 Anne Benoit,Fanny Dufoss´ e,Yves Robert () Mapping filter services on heterogeneous platforms January 9, 2009 8 / 42

Example c 1 = 1, c 2 = 4, c 3 = 10 σ 1 = 1 2 , σ 2 = σ 3 = 1 3 s 1 = 1, s 2 = 2 and s 3 = 3 C 1 C 1 C 2 C 3 C 3 C 2 Figure: Optimal plan for period. Figure: Optimal plan for latency L = 13 P = 1 6 L = 5 P = 4 2 3 Anne Benoit,Fanny Dufoss´ e,Yves Robert () Mapping filter services on heterogeneous platforms January 9, 2009 8 / 42

Framework 1 Period 2 General structure of optimal solutions Case of homogeneous servers NP-completeness of MinPeriod - Het Integer linear program Latency 3 General structure of optimal solutions Polynomial algorithm on homogeneous platforms NP-completeness of problem MinLatency - NoPrec - Het Integer linear program Bi-criteria problem 4 Heuristics 5 Experiments 6 Conclusion 7 Anne Benoit,Fanny Dufoss´ e,Yves Robert () Mapping filter services on heterogeneous platforms January 9, 2009 9 / 42

General structure of optimal solutions The instance : C 1 , ..., C n , S 1 , ..., S n with σ 1 , ..., σ p ≤ 1 σ p +1 , ..., σ n ≥ 1 Anne Benoit,Fanny Dufoss´ e,Yves Robert () Mapping filter services on heterogeneous platforms January 9, 2009 10 / 42

General structure of optimal solutions The instance : C 1 , ..., C n , S 1 , ..., S n with σ 1 , ..., σ p ≤ 1 σ p +1 , ..., σ n ≥ 1 C p +1 C λ (1) C λ (2) C λ (3) C λ ( p ) C n Figure: General structure Anne Benoit,Fanny Dufoss´ e,Yves Robert () Mapping filter services on heterogeneous platforms January 9, 2009 10 / 42

Homogeneous case without precedence constraints The instance : C 1 , ..., C n with c 1 ≤ c 2 ≤ ... ≤ c p σ 1 , ..., σ p < 1 σ p +1 , ..., σ n ≥ 1 The matching: C 1 → C 2 → ... → C p Anne Benoit,Fanny Dufoss´ e,Yves Robert () Mapping filter services on heterogeneous platforms January 9, 2009 11 / 42

Homogeneous case with precedence constraints Computing the optimal subgraph for C in the graph G : Let D = max i { log σ i } . We construct a network flow graph W with: a source s a node f i by service in G a sink node t an edge s − > f i with capacity + ∞ if C i is ancestor of C in G , D else an edge f i − > f j of capacity + ∞ if C j is an ancestor of C i in G an edge f i − > t with capacity D + log σ i The set of services on the side of s in a min-cut is the optimal subset of predecessors for latency. Anne Benoit,Fanny Dufoss´ e,Yves Robert () Mapping filter services on heterogeneous platforms January 9, 2009 12 / 42

Homogeneous case with precedence constraints Computing the optimal subgraph for C in the graph G : Let D = max i { log σ i } . We construct a network flow graph W with: a source s a node f i by service in G a sink node t an edge s − > f i with capacity + ∞ if C i is ancestor of C in G , D else an edge f i − > f j of capacity + ∞ if C j is an ancestor of C i in G an edge f i − > t with capacity D + log σ i The set of services on the side of s in a min-cut is the optimal subset of predecessors for latency. Optimal algorithm: at each step place the available service with minimal possible period. Anne Benoit,Fanny Dufoss´ e,Yves Robert () Mapping filter services on heterogeneous platforms January 9, 2009 12 / 42

Framework 1 Period 2 General structure of optimal solutions Case of homogeneous servers NP-completeness of MinPeriod - Het Integer linear program Latency 3 General structure of optimal solutions Polynomial algorithm on homogeneous platforms NP-completeness of problem MinLatency - NoPrec - Het Integer linear program Bi-criteria problem 4 Heuristics 5 Experiments 6 Conclusion 7 Anne Benoit,Fanny Dufoss´ e,Yves Robert () Mapping filter services on heterogeneous platforms January 9, 2009 13 / 42

Proof: NP-completeness of MinPeriod - Het Problem ( RN3DM ) Given an integer vector A = ( A [1] , . . . , A [ n ]) of size n, does there exist two permutations λ 1 and λ 2 of { 1 , 2 , . . . , n } such that ∀ 1 ≤ i ≤ n , λ 1 ( i ) + λ 2 ( i ) = A [ i ] Anne Benoit,Fanny Dufoss´ e,Yves Robert () Mapping filter services on heterogeneous platforms January 9, 2009 14 / 42

Proof: NP-completeness of MinPeriod - Het Problem ( RN3DM ) Given an integer vector A = ( A [1] , . . . , A [ n ]) of size n, does there exist two permutations λ 1 and λ 2 of { 1 , 2 , . . . , n } such that ∀ 1 ≤ i ≤ n , λ 1 ( i ) + λ 2 ( i ) = A [ i ] The associated instance : c i = 2 A [ i ] σ i = 1 / 2 s i = 2 i P = 2 Anne Benoit,Fanny Dufoss´ e,Yves Robert () Mapping filter services on heterogeneous platforms January 9, 2009 14 / 42

Proof: NP-completeness of MinPeriod - Het Problem ( RN3DM ) Given an integer vector A = ( A [1] , . . . , A [ n ]) of size n, does there exist two permutations λ 1 and λ 2 of { 1 , 2 , . . . , n } such that ∀ 1 ≤ i ≤ n , λ 1 ( i ) + λ 2 ( i ) = A [ i ] The associated instance : c i = 2 A [ i ] σ i = 1 / 2 s i = 2 i P = 2 ∀ 1 ≤ i ≤ n , λ 1 ( i ) + λ 2 ( i ) ≥ A [ i ] � 1 � λ 1 ( i ) − 1 × 2 A [ i ] ⇐ ⇒ ∀ 1 ≤ i ≤ n , 2 λ 2( i ) ≤ 2 2 Anne Benoit,Fanny Dufoss´ e,Yves Robert () Mapping filter services on heterogeneous platforms January 9, 2009 14 / 42

Inapproximability of MinPeriod - Het Proposition For any K > 0 , there exists no K-approximation algorithm for MinPeriod - NoPrec - Het , unless P=NP. Anne Benoit,Fanny Dufoss´ e,Yves Robert () Mapping filter services on heterogeneous platforms January 9, 2009 15 / 42

Inapproximability of MinPeriod - Het Proposition For any K > 0 , there exists no K-approximation algorithm for MinPeriod - NoPrec - Het , unless P=NP. Reduction from RN3DM : c i = K A [ i ] − 1 σ i = 1 / K s i = K i P = 1 Anne Benoit,Fanny Dufoss´ e,Yves Robert () Mapping filter services on heterogeneous platforms January 9, 2009 15 / 42

Integer linear program The variables: t i , u = 1 if service C i is assigned to server S u s i , j = 1 if service C i is an ancestor of C j M is the logarithm of the optimal period Anne Benoit,Fanny Dufoss´ e,Yves Robert () Mapping filter services on heterogeneous platforms January 9, 2009 16 / 42