On the Impact of Platform Models EPIT 2007 Arnaud Legrand - PowerPoint PPT Presentation

Single-port (Full-Duplex) This model somehow makes sense when using networks like Myrinet that have few multiplexing units and with protocols without control flow [Mar07]. Even if it does not model well complex situations, such a model is not harmfull. A. Legrand (CNRS) INRIA-MESCAL Modeling Concurency 19 / 108 On the Impact of Platform Models

Fluid Modeling When using TCP-based networks, it is generally reasonnable to use flows to model bandwidth sharing [MR99, Low03]. � Income Maximization maximize ρ r ∀ l ∈ L , � r ∈R ρ r � c l Max-Min Fairness maximize min r ∈R ρ r r ∈R s.t. l ∈ r � Proportional Fairness maximize log( ρ r ) r ∈R 1 � Potential Delay Minimization minimize ρ r r ∈R � Some weird function minimize arctan( ρ r ) r ∈R A. Legrand (CNRS) INRIA-MESCAL Modeling Concurency 20 / 108 On the Impact of Platform Models

Fluid Modeling When using TCP-based networks, it is generally reasonnable to use flows to model bandwidth sharing [MR99, Low03]. � Income Maximization maximize ρ r ∀ l ∈ L , � r ∈R ρ r � c l Max-Min Fairness maximize min r ∈R ρ r ATM r ∈R s.t. l ∈ r � Proportional Fairness maximize log( ρ r ) r ∈R TCP Vegas 1 � Potential Delay Minimization minimize ρ r r ∈R � Some weird function minimize arctan( ρ r ) r ∈R TCP Reno A. Legrand (CNRS) INRIA-MESCAL Modeling Concurency 20 / 108 On the Impact of Platform Models

Flows Extensions ◮ Note that this model is a multi-port model with capacity-constraints (like in the previous bounded multi-port). ◮ When latencies are large, using multiple connections enables to get more bandwidth. As a matter of fact, there is very few to loose in using multiple connections. . . ◮ Therefore many people enforce a sometimes artificial (but less intrusive) bound on the maximum number of connections per link [Wag05, MYCR06]. A. Legrand (CNRS) INRIA-MESCAL Modeling Concurency 21 / 108 On the Impact of Platform Models

Outline Topology 1 Point to Point Communication Models 2 Modeling Concurency 3 Remind This is a Model, Hence Imperfect 4 A. Legrand (CNRS) INRIA-MESCAL Imperfection 22 / 108 On the Impact of Platform Models

Remind This is a Model, Hence Imperfect ◮ The previous sharing models are nice but you generally do not know other flows. . . ◮ Communications use the memory bus and hence interfere with computations. Taking such interferences into account may be- come more and more important with multi-core architectures. ◮ Interference between communications are sometimes. . . surprising. Modeling is an art. You have to know your platform and your ap- plication to know what is negligeable and what is important. Even if your model is imperfect, you may still derive interesting results. A. Legrand (CNRS) INRIA-MESCAL Imperfection 23 / 108 On the Impact of Platform Models

Part II Scheduling Case Study A. Legrand (CNRS) INRIA-MESCAL 24 / 108 On the Impact of Platform Models

Outline Scheduling Divisible Workload 5 Star-like Network Under the Multi-port Model Bus-like Network Star-like Network Under the One-Port Model Multi-round algorithms Iterative Algorithms 6 Data Redistribution 7 A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 25 / 108 On the Impact of Platform Models

Context: Distributed Heterogeneous Platforms Scheduling divisible load on various architectures [Rob, BGMR96, Bea05, Yan07]. Sources of problems ◮ Point to point communication model (homogeneous/heterogeneous, with or without latency,. . . ) ◮ Concurrency impact. A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 26 / 108 On the Impact of Platform Models

Seismic Tomography of the Earth ◮ Model of the inner structure of the Earth ◮ The model is validated by comparing the propagation time of a seismic wave in the model to the actual propagation time. ◮ Set of all seismic events of the year 1999: 817101 ◮ Original program written for a parallel computer: if (rank = ROOT) raydata ← read n lines from data file; MPI Scatter(raydata, n/P , ..., rbuff, ..., ROOT, MPI COMM WORLD); compute work(rbuff); A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 27 / 108 On the Impact of Platform Models

Applications Covered by The Divisible Loads Model Applications made of a very (very) large number of fine grain computations. Computation time proportional to the size of the data to be processed. Independent computations: neither synchronizations nor communications. A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 28 / 108 On the Impact of Platform Models

Star-like Network �� ◮ The links between the master and the workers have different characteristics. ◮ The workers have different computational power. ◮ Communications from the master to the workers can be done in parallel. A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 30 / 108 On the Impact of Platform Models

Notations ◮ A set P 1 , ..., P p of processors ◮ P 1 is the master processor: initially, it holds all the data. ◮ The overall amount of work: W total . ◮ Processor P i receives an amount of work α i W total with α i ∈ Q and � i α i = 1 . Length of a unit-size work on processor P i : w i . Computation time on P i : α i W total w i . ◮ Time needed to send a unit-message from P 1 to P i : c i . Communication time on P i : α i W total c i . Multi-port model: P 1 can send messages in parallel to all workers. A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 31 / 108 On the Impact of Platform Models

“Optimization” Problem If all communications start in parallel at time 0 , the completion time T i of processor P i is equal to: T i = α i W total c i + α i W total w i The makespan T of a load distribution is thus equal to: max α i W total ( c i + w i ) = T i Therefore this problem is really trivial as we just need to note that α i = T/ ( W total ( c i + w i )) and � i α j = 1 to get T . Hence, we minimize the makespan by setting: P 0 P 1 P 2 P 3 P 4 P 5 1 α i = c i + w i � j c j + w j A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 32 / 108 On the Impact of Platform Models

Latencies: just for fun Let’s assume that the time needed to send a message of size α i from P 1 to P i is now equal to: L i + c i × α i Therefore in the optimal solution: forall i such that α i > 0 , L i + α i W total × ( c i + w i ) = T . So just sort the processor by increasing latency and “fill” the W total units of fluid load (the “density” of one unit of load on P i being equal to c i + w i ). P 0 P 1 P 2 P 3 P 4 P 5 A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 33 / 108 On the Impact of Platform Models

Notations ◮ A set P 1 , ..., P p of processors ◮ P 1 is the master processor: initially, it holds all the data. ◮ The overall amount of work: W total . ◮ Processor P i receives an amount of work α i W total with α i W total ∈ Q and � i α i = 1 . Length of a unit-size work on processor P i : w i . Computation time on P i : n i w i . ◮ Time needed to send a unit-message from P 1 to P i : c . One-port model: P 1 sends a single message at a time, all processors communicate at the same speed with the master. A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 35 / 108 On the Impact of Platform Models

Equations For processor P i (with c 1 = 0 and c j = c otherwise): i � T i = α j W total .c j + α i W total .w i j =1   i � T = max α j W total .c j + α i W total .w i   1 � i � p j =1 We look for a data distribution α 1 , ..., α p which minimizes T . A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 36 / 108 On the Impact of Platform Models

Properties of Load-Balancing Lemma 1. In an optimal solution, all processors end their processing at the same time. A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 37 / 108 On the Impact of Platform Models

Demonstration of Lemma 1 Two workers i and i + 1 with T i < T i +1 . P 4 P 3 P 2 P 1 temps 0 fin We decrease α i +1 by ε . A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 38 / 108 On the Impact of Platform Models

Demonstration of Lemma 1 Two workers i and i + 1 with T i < T i +1 . P 4 P 3 P 2 P 1 temps 0 fin We increase α i by ε . A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 38 / 108 On the Impact of Platform Models

Demonstration of Lemma 1 Two workers i and i + 1 with T i < T i +1 . P 4 P 3 P 2 P 1 temps 0 fin The communication time for the following processors is unchanged. A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 38 / 108 On the Impact of Platform Models

Demonstration of Lemma 1 Two workers i and i + 1 with T i < T i +1 . P 4 P 3 P 2 P 1 temps 0 fin We end up with a better solution ! A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 38 / 108 On the Impact of Platform Models

Property for the Resource Selection Lemma 2. In an optimal solution all processors work. A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 39 / 108 On the Impact of Platform Models

Property for the Resource Selection Lemma 2. In an optimal solution all processors work. Demonstration: this is just a corollary of lemma 1... A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 39 / 108 On the Impact of Platform Models

Resolution T = α 1 W total w 1 . A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 40 / 108 On the Impact of Platform Models

Resolution T = α 1 W total w 1 . w 1 T = α 2 ( c + w 2 ) W total . Therefore α 2 = c + w 2 α 1 . A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 40 / 108 On the Impact of Platform Models

Resolution T = α 1 W total w 1 . w 1 T = α 2 ( c + w 2 ) W total . Therefore α 2 = c + w 2 α 1 . w 2 T = ( α 2 c + α 3 ( c + w 3 )) W total . Therefore α 3 = c + w 3 α 2 . A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 40 / 108 On the Impact of Platform Models

Resolution T = α 1 W total w 1 . w 1 T = α 2 ( c + w 2 ) W total . Therefore α 2 = c + w 2 α 1 . w 2 T = ( α 2 c + α 3 ( c + w 3 )) W total . Therefore α 3 = c + w 3 α 2 . α i = w i − 1 c + w i α i − 1 for i � 2 . A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 40 / 108 On the Impact of Platform Models

Resolution T = α 1 W total w 1 . w 1 T = α 2 ( c + w 2 ) W total . Therefore α 2 = c + w 2 α 1 . w 2 T = ( α 2 c + α 3 ( c + w 3 )) W total . Therefore α 3 = c + w 3 α 2 . α i = w i − 1 c + w i α i − 1 for i � 2 . � n i =1 α i = 1 . A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 40 / 108 On the Impact of Platform Models

Resolution T = α 1 W total w 1 . w 1 T = α 2 ( c + w 2 ) W total . Therefore α 2 = c + w 2 α 1 . w 2 T = ( α 2 c + α 3 ( c + w 3 )) W total . Therefore α 3 = c + w 3 α 2 . α i = w i − 1 c + w i α i − 1 for i � 2 . � n i =1 α i = 1 . j � � w 1 w k − 1 � α 1 1 + + ... + + ... = 1 c + w 2 c + w k k =2 A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 40 / 108 On the Impact of Platform Models

Impact of the Order of Communications How important is the influence of the ordering of the processor on the solution ? ? A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 41 / 108 On the Impact of Platform Models

No Impact of The Order of the Communications Volume processed by processors P i and P i +1 during a time T . 1 T Processor P i : α i ( c + w i ) W total = T . Therefore α i = W total . c + w i Processor P i +1 : α i cW total + α i +1 ( c + w i +1 ) W total = T . w i 1 T T Thus α i +1 = c + w i +1 ( W total − α i c ) = W total . ( c + w i )( c + w i +1 ) Processors P i and P i +1 : c + w i + w i +1 α i + α i +1 = ( c + w i )( c + w i +1 ) A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 42 / 108 On the Impact of Platform Models

Choice of the Master Processor We compare processors P 1 and P 2 . A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 43 / 108 On the Impact of Platform Models

Choice of the Master Processor We compare processors P 1 and P 2 . 1 T Processor P 1 : α 1 w 1 W total = T . Then, α 1 = W total . w 1 A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 43 / 108 On the Impact of Platform Models

Choice of the Master Processor We compare processors P 1 and P 2 . 1 T Processor P 1 : α 1 w 1 W total = T . Then, α 1 = W total . w 1 1 T Processor P 2 : α 2 ( c + w 2 ) W total = T . Thus, α 2 = W total . c + w 2 A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 43 / 108 On the Impact of Platform Models

Choice of the Master Processor We compare processors P 1 and P 2 . 1 T Processor P 1 : α 1 w 1 W total = T . Then, α 1 = W total . w 1 1 T Processor P 2 : α 2 ( c + w 2 ) W total = T . Thus, α 2 = W total . c + w 2 Total volume processed: α 1 + α 2 = c + w 1 + w 2 w 1 ( c + w 2 ) = c + w 1 + w 2 cw 1 + w 1 w 2 A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 43 / 108 On the Impact of Platform Models

Choice of the Master Processor We compare processors P 1 and P 2 . 1 T Processor P 1 : α 1 w 1 W total = T . Then, α 1 = W total . w 1 1 T Processor P 2 : α 2 ( c + w 2 ) W total = T . Thus, α 2 = W total . c + w 2 Total volume processed: α 1 + α 2 = c + w 1 + w 2 w 1 ( c + w 2 ) = c + w 1 + w 2 cw 1 + w 1 w 2 Minimal when w 1 < w 2 . Master = the most powerfull processor (for computations). A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 43 / 108 On the Impact of Platform Models

Conclusion ◮ Closed-form expressions for the execution time and the distribution of data. ◮ Choice of the master. ◮ The ordering of the processors has no impact. ◮ All processors take part in the work. A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 44 / 108 On the Impact of Platform Models

Star-like Network � � � � �� ◮ The links between the master and the workers have different characteristics. ◮ The workers have different computational power. A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 46 / 108 On the Impact of Platform Models

Notations ◮ A set P 1 , ..., P p of processors ◮ P 1 is the master processor: initially, it holds all the data. ◮ The overall amount of work: W total . ◮ Processor P i receives an amount of work α i W total with � i n i = W total with α i W total ∈ Q and � i α i = 1 . Length of a unit-size work on processor P i : w i . Computation time on P i : n i w i . ◮ Time needed to send a unit-message from P 1 to P i : c i . One-port model: P 1 sends a single message at a time. A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 47 / 108 On the Impact of Platform Models

Star Network and Linear Cost Model Goal : maximize the number of processed tasks within a time-bound T f : � α i . A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 48 / 108 On the Impact of Platform Models

Star Network and Linear Cost Model Goal : maximize the number of processed tasks within a time-bound T f : � α i . Lemma 3. In any optimal solution of the StarLinear problem, all workers participate in the computation, and all processors finish computing simultaneously. A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 48 / 108 On the Impact of Platform Models

Star Network and Linear Cost Model Goal : maximize the number of processed tasks within a time-bound T f : � α i . Lemma 3. In any optimal solution of the StarLinear problem, all workers participate in the computation, and all processors finish computing simultaneously. Lemma 4. An optimal ordering for the StarLinear problem is obtained by serving the workers in the ordering of non decreasing link capacities c i . A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 48 / 108 On the Impact of Platform Models

Sketch of the Proof of Lemma 3 Two steps : ◮ All workers participate in the computation. . . A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 49 / 108 On the Impact of Platform Models

Sketch of the Proof of Lemma 3 P p α p w p ... Two steps : P i ◮ All workers participate in ... the computation. . . otherwise P 2 α 2 w 2 it would not be optimal. P 1 α 1 w 1 Network α 1 c 1 α 2 c 2 α p c p T 1 T 2 T p T f A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 49 / 108 On the Impact of Platform Models

Sketch of the Proof of Lemma 3 P p α p w p ... Two steps : P i α i w i ◮ All workers participate in ... the computation. . . otherwise P 2 α 2 w 2 it would not be optimal. P 1 α 1 w 1 ◮ All Network α 1 c 1 α 2 c 2 α p c p α i c i processors finish their T 1 T 2 T p T f work at the same time. A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 49 / 108 On the Impact of Platform Models

Sketch of the Proof of Lemma 3 P p α p w p ... Two steps : P i α i w i ◮ All workers participate in ... the computation. . . otherwise P 2 α 2 w 2 it would not be optimal. P 1 α 1 w 1 ◮ All Network α 1 c 1 α 2 c 2 ... α i c i ... α p c p processors finish their T 1 T 2 T p T f work at the same time. Maximize � β i , subject to � LB( i ) ∀ i, β i � 0 � i UB( i ) ∀ i, k =1 β k c k + β i w i � T f A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 49 / 108 On the Impact of Platform Models

Sketch of the Proof of Lemma 3 β 2 Two steps : ◮ All workers participate in the computation. . . otherwise it would not be optimal. ◮ All processors finish their work at the same time. β 1 Maximize � β i , subject to � LB( i ) ∀ i, β i � 0 � i UB( i ) ∀ i, k =1 β k c k + β i w i � T f A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 49 / 108 On the Impact of Platform Models

Sketch of the Proof of Lemma 3 β 2 Two steps : ◮ All workers participate in ( α 1 , α 2 ) the computation. . . otherwise it would not be optimal. ◮ All processors finish their work at the same time. β 1 Maximize � β i , subject to � LB( i ) ∀ i, β i � 0 � i UB( i ) ∀ i, k =1 β k c k + β i w i � T f A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 49 / 108 On the Impact of Platform Models

Sketch of the Proof of Lemma 4 The proof is based on the comparison of the amount of work that is performed by the first two workers, and then proceeds by induction. A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 50 / 108 On the Impact of Platform Models

Sketch of the Proof of Lemma 4 The proof is based on the comparison of the amount of work that is performed by the first two workers, and then proceeds by induction. t ( A ) t ( B ) T T α ( A ) α ( A ) α ( B ) α ( B ) P 1 1 c 1 1 w 1 P 1 1 c 1 1 w 1 α ( A ) α ( A ) α ( B ) α ( B ) P 2 P 2 2 c 2 2 w 2 2 c 2 2 w 2 (A) P 1 starts before P 2 (B) P 2 starts before P 1 A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 50 / 108 On the Impact of Platform Models

Sketch of the Proof of Lemma 4 The proof is based on the comparison of the amount of work that is performed by the first two workers, and then proceeds by induction. t ( A ) t ( B ) T T α ( A ) α ( A ) α ( B ) α ( B ) P 1 1 c 1 1 w 1 P 1 1 c 1 1 w 1 α ( A ) α ( A ) α ( B ) α ( B ) P 2 P 2 2 c 2 2 w 2 2 c 2 2 w 2 (A) P 1 starts before P 2 (B) P 2 starts before P 1 t ( A ) = t ( B ) , (1) A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 50 / 108 On the Impact of Platform Models

Sketch of the Proof of Lemma 4 The proof is based on the comparison of the amount of work that is performed by the first two workers, and then proceeds by induction. t ( A ) t ( B ) T T α ( A ) α ( A ) α ( B ) α ( B ) P 1 1 c 1 1 w 1 P 1 1 c 1 1 w 1 α ( A ) α ( A ) α ( B ) α ( B ) P 2 P 2 2 c 2 2 w 2 2 c 2 2 w 2 (A) P 1 starts before P 2 (B) P 2 starts before P 1 t ( A ) = t ( B ) , (1) and T ( c 2 − c 1 ) ( α ( A ) + α ( A ) ) − ( α ( B ) + α ( B ) ) = ( c 1 + w 1 )( c 2 + w 2 ) . (2) 1 2 1 2 A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 50 / 108 On the Impact of Platform Models

Conclusion ◮ The processors must be ordered by decreasing bandwidths ◮ All processors are working ◮ All processors end their work at the same time ◮ Formulas for the execution time and the distribution of data A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 51 / 108 On the Impact of Platform Models

One-round vs. Multi-round P p α p w p P p ... P 2 α 2 w 2 P 2 P 1 α 1 w 1 P 1 Network α 1 g α 2 g α p g Network T 1 T 2 T p T f R 0 R 1 R k One round Multi-round A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 53 / 108 On the Impact of Platform Models

One-round vs. Multi-round P p α p w p P p ... P 2 α 2 w 2 P 2 P 1 α 1 w 1 P 1 Network α 1 g α 2 g α p g Network T 1 T 2 T p T f R 0 R 1 R k One round Multi-round � long idle-times Efficient when W total large Intuition: start with small rounds, then increase chunks. Problems: A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 53 / 108 On the Impact of Platform Models

One-round vs. Multi-round P p α p w p P p ... P 2 α 2 w 2 P 2 P 1 α 1 w 1 P 1 Network α 1 g α 2 g α p g Network T 1 T 2 T p T f R 0 R 1 R k One round Multi-round � long idle-times Efficient when W total large Intuition: start with small rounds, then increase chunks. Problems: ◮ linear communication model leads to absurd solution ◮ resource selection ◮ number of rounds ◮ size of each round A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 53 / 108 On the Impact of Platform Models

Notations ◮ A set P 1 , ..., P p of processors ◮ P 1 is the master processor: initially, it holds all the data. ◮ The overall amount of work: W total . ◮ Processor P i receives an amount of work α i W total with � i n i = W total with α i W total ∈ Q and � i α i = 1 . Length of a unit-size work on processor P i : w i . Computation time on P i : n i w i . ◮ Time needed to send a message of size α i P 1 to P i : L i + c i × α i . One-port model: P 1 sends and receives a single message at a time. A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 54 / 108 On the Impact of Platform Models

Complexity Definition: One round, ∀ i, c i = 0 . Given W total , p workers, ( P i ) 1 � i � p , ( L i ) 1 � i � p , and a rational number T � 0 , and assuming that bandwidths are infinite, is it possible to compute all W total load units within T time units? Theorem 1. The problem with one-round and infinite bandwidths is NP- complete. A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 55 / 108 On the Impact of Platform Models

Complexity Definition: One round, ∀ i, c i = 0 . Given W total , p workers, ( P i ) 1 � i � p , ( L i ) 1 � i � p , and a rational number T � 0 , and assuming that bandwidths are infinite, is it possible to compute all W total load units within T time units? Theorem 1. The problem with one-round and infinite bandwidths is NP- complete. What is the complexity of the general problem with finite bandwidths and several rounds ? A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 55 / 108 On the Impact of Platform Models

Complexity Definition: One round, ∀ i, c i = 0 . Given W total , p workers, ( P i ) 1 � i � p , ( L i ) 1 � i � p , and a rational number T � 0 , and assuming that bandwidths are infinite, is it possible to compute all W total load units within T time units? Theorem 1. The problem with one-round and infinite bandwidths is NP- complete. What is the complexity of the general problem with finite bandwidths and several rounds ? The general problem is NP-hard, but does not appear to be in NP (no polynomial bound on the number of activations). A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 55 / 108 On the Impact of Platform Models

Fixed activation sequence Hypotheses 1 Number of activations : N act ; 2 Whether P i is the processor used during activation j : χ ( j ) i Minimize T, under the constraints  p N act χ ( j ) i α ( j )  � � = W total   i    j =1 i =1      p k N act � � χ ( j ) i ( L i + α ( j ) � χ ( j ) l α ( j )  + ∀ k � N act , ∀ l : i c i ) l w l � T      j =1 i =1 j = k    ∀ i, j : α ( j )  � 0  i (3) Can be solved in polynomial time. A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 56 / 108 On the Impact of Platform Models

Fixed number of activations Minimize T, under the constraints p N act  χ ( j ) i α ( j ) � �  = W total   i    j =1 i =1      k p N act   χ ( j ) i ( L i + α ( j ) χ ( j ) l α ( j )  � �  + �  ∀ k � N act , ∀ l : i c i ) l w l � T     j =1 i =1 j = k p  � χ ( k )  ∀ k � N act : � 1   i    i =1   ∀ i, j : χ ( j )  ∈ { 0 , 1 }   i   ∀ i, j : α ( j )   � 0 i (4) Exact but exponential (branch-and-bound algorithms). A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 57 / 108 On the Impact of Platform Models

Uniform Multi-Round round j round j + 1 round j + 2 α ( j ) α ( j +1) 1 c i c i 1 Transfer Worker 1 In a round: all workers have same α ( j ) 1 w 1 Compute computation time Transfer Worker 2 Compute . . α ( j ) i c i α ( j +1) c i . i Geometrical increase of rounds Transfer Worker i α ( j ) i w i = α ( j ) 1 w 1 Compute size L i . . α ( j ) α ( j +1) p c p c p . p No idle time in communications: Transfer Worker p α ( j ) p w p = α ( j ) 1 w 1 Compute T A T C T B time p α ( j ) � ( L k + α ( j +1) i w i = c k ) . k k =1 Heuristic processor selection: by decreasing bandwidths A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 58 / 108 On the Impact of Platform Models

Uniform Multi-Round round j round j + 1 round j + 2 α ( j ) α ( j +1) 1 c i c i 1 Transfer Worker 1 In a round: all workers have same α ( j ) 1 w 1 Compute computation time Transfer Worker 2 Compute . . α ( j ) i c i α ( j +1) c i . i Geometrical increase of rounds Transfer Worker i α ( j ) i w i = α ( j ) 1 w 1 Compute size L i . . α ( j ) α ( j +1) p c p c p . p No idle time in communications: Transfer Worker p α ( j ) p w p = α ( j ) 1 w 1 Compute T A T C T B time p α ( j ) � ( L k + α ( j +1) i w i = c k ) . k k =1 Heuristic processor selection: by decreasing bandwidths No guarantee. . . A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 58 / 108 On the Impact of Platform Models

Periodic Schedule T p L 1 α 1 c 1 L 1 α 1 c 1 L 1 α 1 c 1 Transfer α 1 w 1 α 1 w 1 α 1 w 1 Compute L 2 α 2 c 2 L 2 α 2 c 2 L 2 α 2 c 2 Transfer α 2 w 2 α 2 w 2 α 2 w 2 Compute L 3 α 3 c 3 L 3 α 3 c 3 L 3 α 3 c 3 Transfer α 3 w 3 α 3 w 3 α 3 w 3 Compute . . . L n α n c n L n α n c n L n α n c n Transfer α n w n α n w n α n w n Compute How to choose T p ? Which resources to select? A. Legrand (CNRS) INRIA-MESCAL Divisible Workload 59 / 108 On the Impact of Platform Models

On the Impact of Platform Models EPIT 2007 Arnaud Legrand - PowerPoint PPT Presentation

On the Impact of Platform Models EPIT 2007 Arnaud Legrand CNRS/INRIA, LIG laboratory June 6, 2007 A. Legrand (CNRS) INRIA-MESCAL 1 / 108 On the Impact of Platform Models Motivation Scientific computing : large needs in computation or

Automated Generation of Platform-Variant Applications from Platform-Independent Models via

Ride Sharing Platform Vs Taxi Platform: the Impact on the Revenue Benjamin Bordais 1 , Costas

Logic Models Defining Impact and Change 1 28/06/2013 One Tool, Many Names Logic Models or

The Impact of Artificial Intelligence on Naval Platform and System Design C.D.Burnside Rotterdam

Computer Models of Micrometeoroid Impact on Fused Silica Glass Mirrors Hypervelocity Impact

Future of Cultural Heritage Impact of external developments European Foresight Platform Workshop

Impact of Pharmacovigilance Systems 4 th industry stakeholder platform - operation of EU

A multi-platform system for understanding, monitoring and forecasting the impact of aerosol

The Web of Mathematical Models: A Schema-based, Wiki-like, Interactive Platform Thomas

Neutrino Flavour Models and Impact on LFV Luca Merlo 26.06.2012,

1 SIP History Societal impact of pain (SIP) is a social platform created in 2009 which

UDT 2020 The Impact of Artificial Intelligence on Naval Platform and System Design

NSML : A Machine Learning Platform That Enables You Focus on Your Models. ML-Sys WS 2017 @ NIPS

AutonoVi-Sim: Modular Autonomous Vehicle Simulation Platform Supporting Diverse Vehicle Models,

Platform-Independent Debugging of Physical Interaction and Signal Flow Models Mehdi Dadfarnia 1

Delineating the Impact of Weightlessness on Human Physiology Using Computational Models Mohammad

Examining the Impact of Prandtl Number and Heat Transport Models on

Forbruger Europa Christel Hst clp@forbrugereuropa.dk ODR platform About the ODR-platform

Beyond uber how platforms rule the world Matthias Walter T-Systems MMS The platform innovation

Delfi COM Platform What stands behind the term: Delfi COM Platform? The Delfi COM Platform

Capsis A software platform to host various kinds of growth and yield or forest dynamics models

Presentation of Platform Zero Incidents Platform Zero Incidents Platform Zero Incidents MENTAL

TREE = TOKEN The Frontier of Impact Finance T TREE T TREE Token = oken = 1 The Frontier

o Impact of different Geant4 effects and Genfit MSC models on Si-tracking Moritz Nadler