Mapping pipeline skeletons onto heterogeneous platforms Anne Benoit - PowerPoint PPT Presentation

Introduction Framework Complexity Heuristics Experiments Conclusion Mapping pipeline skeletons onto heterogeneous platforms Anne Benoit and Yves Robert GRAAL team, LIP ´ Ecole Normale Sup´ erieure de Lyon May 2007 Anne.Benoit@ens-lyon.fr May 2007 Mapping pipeline skeletons ICCS’07 1/ 26

Introduction Framework Complexity Heuristics Experiments Conclusion Introduction and motivation Mapping applications onto parallel platforms Difficult challenge Heterogeneous clusters, fully heterogeneous platforms Even more difficult! Structured programming approach Easier to program (deadlocks, process starvation) Range of well-known paradigms (pipeline, farm) Algorithmic skeleton: help for mapping Mapping pipeline skeletons onto heterogeneous platforms Anne.Benoit@ens-lyon.fr May 2007 Mapping pipeline skeletons ICCS’07 2/ 26

Introduction Framework Complexity Heuristics Experiments Conclusion Introduction and motivation Mapping applications onto parallel platforms Difficult challenge Heterogeneous clusters, fully heterogeneous platforms Even more difficult! Structured programming approach Easier to program (deadlocks, process starvation) Range of well-known paradigms (pipeline, farm) Algorithmic skeleton: help for mapping Mapping pipeline skeletons onto heterogeneous platforms Anne.Benoit@ens-lyon.fr May 2007 Mapping pipeline skeletons ICCS’07 2/ 26

Introduction Framework Complexity Heuristics Experiments Conclusion Introduction and motivation Mapping applications onto parallel platforms Difficult challenge Heterogeneous clusters, fully heterogeneous platforms Even more difficult! Structured programming approach Easier to program (deadlocks, process starvation) Range of well-known paradigms (pipeline, farm) Algorithmic skeleton: help for mapping Mapping pipeline skeletons onto heterogeneous platforms Anne.Benoit@ens-lyon.fr May 2007 Mapping pipeline skeletons ICCS’07 2/ 26

Introduction Framework Complexity Heuristics Experiments Conclusion Introduction and motivation Mapping applications onto parallel platforms Difficult challenge Heterogeneous clusters, fully heterogeneous platforms Even more difficult! Structured programming approach Easier to program (deadlocks, process starvation) Range of well-known paradigms (pipeline, farm) Algorithmic skeleton: help for mapping Mapping pipeline skeletons onto heterogeneous platforms Anne.Benoit@ens-lyon.fr May 2007 Mapping pipeline skeletons ICCS’07 2/ 26

Introduction Framework Complexity Heuristics Experiments Conclusion Introduction and motivation Mapping applications onto parallel platforms Difficult challenge Heterogeneous clusters, fully heterogeneous platforms Even more difficult! Structured programming approach Easier to program (deadlocks, process starvation) Range of well-known paradigms (pipeline, farm) Algorithmic skeleton: help for mapping Mapping pipeline skeletons onto heterogeneous platforms Anne.Benoit@ens-lyon.fr May 2007 Mapping pipeline skeletons ICCS’07 2/ 26

Introduction Framework Complexity Heuristics Experiments Conclusion Introduction and motivation Mapping applications onto parallel platforms Difficult challenge Heterogeneous clusters, fully heterogeneous platforms Even more difficult! Structured programming approach Easier to program (deadlocks, process starvation) Range of well-known paradigms (pipeline, farm) Algorithmic skeleton: help for mapping Mapping pipeline skeletons onto heterogeneous platforms Anne.Benoit@ens-lyon.fr May 2007 Mapping pipeline skeletons ICCS’07 2/ 26

Introduction Framework Complexity Heuristics Experiments Conclusion Introduction and motivation Mapping applications onto parallel platforms Difficult challenge Heterogeneous clusters, fully heterogeneous platforms Even more difficult! Structured programming approach Easier to program (deadlocks, process starvation) Range of well-known paradigms (pipeline, farm) Algorithmic skeleton: help for mapping Mapping pipeline skeletons onto heterogeneous platforms Anne.Benoit@ens-lyon.fr May 2007 Mapping pipeline skeletons ICCS’07 2/ 26

Introduction Framework Complexity Heuristics Experiments Conclusion Rule of the game Map each pipeline stage on a single processor Goal: minimize execution time Several mapping strategies S 1 S 2 S k S n ... ... The pipeline application Anne.Benoit@ens-lyon.fr May 2007 Mapping pipeline skeletons ICCS’07 3/ 26

Introduction Framework Complexity Heuristics Experiments Conclusion Rule of the game Map each pipeline stage on a single processor Goal: minimize execution time Several mapping strategies S 1 S 2 S k S n ... ... The pipeline application Anne.Benoit@ens-lyon.fr May 2007 Mapping pipeline skeletons ICCS’07 3/ 26

Introduction Framework Complexity Heuristics Experiments Conclusion Rule of the game Map each pipeline stage on a single processor Goal: minimize execution time Several mapping strategies S 1 S 2 S k S n ... ... One-to-one Mapping Anne.Benoit@ens-lyon.fr May 2007 Mapping pipeline skeletons ICCS’07 3/ 26

Introduction Framework Complexity Heuristics Experiments Conclusion Rule of the game Map each pipeline stage on a single processor Goal: minimize execution time Several mapping strategies S 1 S 2 S k S n ... ... Interval Mapping Anne.Benoit@ens-lyon.fr May 2007 Mapping pipeline skeletons ICCS’07 3/ 26

Introduction Framework Complexity Heuristics Experiments Conclusion Rule of the game Map each pipeline stage on a single processor Goal: minimize execution time Several mapping strategies S 1 S 2 S k S n ... ... General Mapping Anne.Benoit@ens-lyon.fr May 2007 Mapping pipeline skeletons ICCS’07 3/ 26

Introduction Framework Complexity Heuristics Experiments Conclusion Major contributions Theory Formal approach to the problem Problem complexity Integer linear program for exact resolution Practice Heuristics for Interval Mapping on clusters Experiments to compare heuristics and evaluate their absolute performance Anne.Benoit@ens-lyon.fr May 2007 Mapping pipeline skeletons ICCS’07 4/ 26

Introduction Framework Complexity Heuristics Experiments Conclusion Major contributions Theory Formal approach to the problem Problem complexity Integer linear program for exact resolution Practice Heuristics for Interval Mapping on clusters Experiments to compare heuristics and evaluate their absolute performance Anne.Benoit@ens-lyon.fr May 2007 Mapping pipeline skeletons ICCS’07 4/ 26

Introduction Framework Complexity Heuristics Experiments Conclusion Outline Framework 1 Complexity results 2 Heuristics 3 Experiments 4 Conclusion 5 Anne.Benoit@ens-lyon.fr May 2007 Mapping pipeline skeletons ICCS’07 5/ 26

Introduction Framework Complexity Heuristics Experiments Conclusion The application δ 0 δ 1 δ k − 1 δ k δ n S 1 S 2 S k S n ... ... w 1 w 2 w k w n n stages S k , 1 ≤ k ≤ n S k : receives input of size δ k − 1 from S k − 1 performs w k computations outputs data of size δ k to S k +1 Anne.Benoit@ens-lyon.fr May 2007 Mapping pipeline skeletons ICCS’07 6/ 26

Introduction Framework Complexity Heuristics Experiments Conclusion The platform s in P in P out s out b v , out b in , u P u P v b u , v s u s v p processors P u , 1 ≤ u ≤ p, fully interconnected s u : speed of processor P u bidirectional link link u , v : P u → P v , bandwidth b u , v one-port model: each processor can either send, receive or compute at any time-step Anne.Benoit@ens-lyon.fr May 2007 Mapping pipeline skeletons ICCS’07 7/ 26

Introduction Framework Complexity Heuristics Experiments Conclusion Different platforms Fully Homogeneous – Identical processors (s u = s) and links (b u , v = b): typical parallel machines Communication Homogeneous – Different-speed processors (s u � = s v ), identical links (b u , v = b): networks of workstations, clusters Fully Heterogeneous – Fully heterogeneous architectures, s u � = s v and b u , v � = b u ′ , v ′ : hierarchical platforms, grids Anne.Benoit@ens-lyon.fr May 2007 Mapping pipeline skeletons ICCS’07 8/ 26

Introduction Framework Complexity Heuristics Experiments Conclusion Mapping problem: Interval Mapping Several consecutive stages onto the same processor Increase computational load, reduce communications Partition of [1 .. n] into m intervals I j = [ d j , e j ] (with d j ≤ e j for 1 ≤ j ≤ m , d 1 = 1, d j +1 = e j + 1 for 1 ≤ j ≤ m − 1 and e m = n) Interval I j mapped onto processor P alloc( j ) Minimize the period: � e j � � i = d j w i δ d j − 1 δ e j T period = max + + b alloc( j − 1) , alloc( j ) s alloc( j ) b alloc( j ) , alloc( j +1) 1 ≤ j ≤ m Anne.Benoit@ens-lyon.fr May 2007 Mapping pipeline skeletons ICCS’07 9/ 26

Introduction Framework Complexity Heuristics Experiments Conclusion Mapping problem: Interval Mapping Several consecutive stages onto the same processor Increase computational load, reduce communications Partition of [1 .. n] into m intervals I j = [ d j , e j ] (with d j ≤ e j for 1 ≤ j ≤ m , d 1 = 1, d j +1 = e j + 1 for 1 ≤ j ≤ m − 1 and e m = n) Interval I j mapped onto processor P alloc( j ) Minimize the period: � e j � � i = d j w i δ d j − 1 δ e j T period = max + + b alloc( j − 1) , alloc( j ) s alloc( j ) b alloc( j ) , alloc( j +1) 1 ≤ j ≤ m Anne.Benoit@ens-lyon.fr May 2007 Mapping pipeline skeletons ICCS’07 9/ 26