Design and Implemention of a Plugin Scheduler for Diet & - PowerPoint PPT Presentation

Background on Diet Plugin Scheduler CoRI Conclusion & Future Works Design and Implemention of a Plugin Scheduler for Diet & Performance Prediction in Diet with CoRI Collectors of Resource Information February 19, 2006 Design and Implemention of a Plugin Scheduler for Diet & Perfo

Background on Diet Plugin Scheduler CoRI Conclusion & Future Works Outline Background on Diet 1 Diet Framework Motivation for Plugin Scheduler Plugin Scheduler 2 Design Implementation CoRI 3 Conclusion & Future Works 4 Design and Implemention of a Plugin Scheduler for Diet & Perfo

Background on Diet Plugin Scheduler Diet Framework CoRI Motivation for Plugin Scheduler Conclusion & Future Works Outline Background on Diet 1 Diet Framework Motivation for Plugin Scheduler Plugin Scheduler 2 Design Implementation CoRI 3 Conclusion & Future Works 4 Design and Implemention of a Plugin Scheduler for Diet & Perfo

Background on Diet Plugin Scheduler Diet Framework CoRI Motivation for Plugin Scheduler Conclusion & Future Works Grids and Diet Grid platforms Heterogeneous computational resources Irregular network topologies Dynamic resource performance Diet philosophy and design principles Server and broker agent model Hierarchical organization Flexible deployment options Design and Implemention of a Plugin Scheduler for Diet & Perfo

Background on Diet Plugin Scheduler Diet Framework CoRI Motivation for Plugin Scheduler Conclusion & Future Works Diet Overview Basic progress of a Diet call: Client requests service from the Diet hierarchy: Master Agent (MA) The MA interrogates the MA Diet hierarchy Each Server Daemon (SeD) responds . . . LA 1 with a response profile Each Local Agent (LA) compiles and sorts the responses by execution time . . . SeD 1 SeD 2 SeD 3 MA returns a list of servers to the client Client launches service directly on SeD SeD response (DIET_profile_t) Problem COMP_TIME t comp t comp Non-standard application- and COMM_TIME t comm platform-specific performance TOTAL_TIME t total AVAIL_MEM m avail measures Design and Implemention of a Plugin Scheduler for Diet & Perfo

Background on Diet Plugin Scheduler Diet Framework CoRI Motivation for Plugin Scheduler Conclusion & Future Works Application-specific Performance Use Case Motivation Basic Diet deployment Diet hierarchy: MA . . . LA 1 . . . SeD 1 SeD 2 SeD 3 Design and Implemention of a Plugin Scheduler for Diet & Perfo

Background on Diet Plugin Scheduler Diet Framework CoRI Motivation for Plugin Scheduler Conclusion & Future Works Application-specific Performance Use Case Motivation Basic Diet deployment Diet hierarchy: Client application with data dependencies MA . . . LA 1 . . . SeD 1 SeD 2 SeD 3 Design and Implemention of a Plugin Scheduler for Diet & Perfo

Background on Diet Plugin Scheduler Diet Framework CoRI Motivation for Plugin Scheduler Conclusion & Future Works Application-specific Performance Use Case Motivation Basic Diet deployment Diet hierarchy: Client application with data dependencies MA . . . LA 1 . . . SeD 1 SeD 2 SeD 3 Design and Implemention of a Plugin Scheduler for Diet & Perfo

Background on Diet Plugin Scheduler Diet Framework CoRI Motivation for Plugin Scheduler Conclusion & Future Works Application-specific Performance Use Case Motivation Basic Diet deployment Diet hierarchy: Client application with data dependencies MA “performance” is not well-defined . . . LA 1 ? ? ? . . . SeD 1 SeD 2 SeD 3 Design and Implemention of a Plugin Scheduler for Diet & Perfo

Background on Diet Plugin Scheduler Diet Framework CoRI Motivation for Plugin Scheduler Conclusion & Future Works Application-specific Performance Use Case Motivation Basic Diet deployment Diet hierarchy: Client application with data dependencies MA “performance” is not well-defined . . . Possible meanings for performance LA 1 Existence of data ? ? ? Avail. free memory . . . SeD 1 SeD 2 SeD 3 Specific architecture Previous scheduling decisions Application-specific measures Composite requirements ... Design and Implemention of a Plugin Scheduler for Diet & Perfo

Background on Diet Plugin Scheduler Diet Framework CoRI Motivation for Plugin Scheduler Conclusion & Future Works Application-specific Performance Use Case Motivation Basic Diet deployment Diet hierarchy: Client application with data dependencies MA “performance” is not well-defined . . . Possible meanings for performance LA 1 Existence of data (GriPPS) ? ? ? Avail. free memory (MUMPS?) . . . SeD 1 SeD 2 SeD 3 Specific architecture (TLSE) Previous scheduling decisions Application-specific measures Composite requirements ... Context-sensitive performance metrics are needed Design and Implemention of a Plugin Scheduler for Diet & Perfo

Background on Diet Plugin Scheduler Design CoRI Implementation Conclusion & Future Works Outline Background on Diet 1 Diet Framework Motivation for Plugin Scheduler Plugin Scheduler 2 Design Implementation CoRI 3 Conclusion & Future Works 4 Design and Implemention of a Plugin Scheduler for Diet & Perfo

Background on Diet Plugin Scheduler Design CoRI Implementation Conclusion & Future Works Plugin Scheduling Plugin scheduling facilities to enable Application-specific definitions of appropriate performance metrics An extensible measurement system Tunable comparison/aggregation routines for scheduling Design and Implemention of a Plugin Scheduler for Diet & Perfo

Background on Diet Plugin Scheduler Design CoRI Implementation Conclusion & Future Works Plugin Scheduling Plugin scheduling facilities to enable Application-specific definitions of appropriate performance metrics An extensible measurement system Tunable comparison/aggregation routines for scheduling Design changes Component Before After automatic performance esti- chosen/defined by applica- SeD mate (FAST/NWS) tion programmer Design and Implemention of a Plugin Scheduler for Diet & Perfo

Background on Diet Plugin Scheduler Design CoRI Implementation Conclusion & Future Works Plugin Scheduling Plugin scheduling facilities to enable Application-specific definitions of appropriate performance metrics An extensible measurement system Tunable comparison/aggregation routines for scheduling Design changes Component Before After automatic performance esti- chosen/defined by applica- SeD mate (FAST/NWS) tion programmer “menu” of aggregation Agents exec. time sorting methods Design and Implemention of a Plugin Scheduler for Diet & Perfo

Background on Diet Plugin Scheduler Design CoRI Implementation Conclusion & Future Works Plugin Scheduling Plugin scheduling facilities to enable Application-specific definitions of appropriate performance metrics An extensible measurement system Tunable comparison/aggregation routines for scheduling Design changes Component Before After automatic performance esti- chosen/defined by applica- SeD mate (FAST/NWS) tion programmer “menu” of aggregation Agents exec. time sorting methods Client CLIENT CODE UNCHANGED Design and Implemention of a Plugin Scheduler for Diet & Perfo

Background on Diet Plugin Scheduler Design CoRI Implementation Conclusion & Future Works Plugin Scheduling Enhancements Example: Client request for comparison operation on blue database or Juxmem repository Request arrives at SeD level Diet hierarchy: MA . . . LA 1 ? ? ? . . . SeD 1 SeD 2 SeD 3 Design and Implemention of a Plugin Scheduler for Diet & Perfo

Background on Diet Plugin Scheduler Design CoRI Implementation Conclusion & Future Works Plugin Scheduling Enhancements Example: Client request for comparison operation on blue database or Juxmem repository Request arrives at SeD level Diet hierarchy: Profile is an esimation vector : dynamic array of (tag,value) pairs MA . . . LA 1 ? ? ? SeD response (DIET_profile_t) . . . SeD 1 SeD 2 SeD 3 EST Design and Implemention of a Plugin Scheduler for Diet & Perfo

Background on Diet Plugin Scheduler Design CoRI Implementation Conclusion & Future Works Plugin Scheduling Enhancements Example: Client request for comparison operation on blue database or Juxmem repository Request arrives at SeD level Diet hierarchy: Profile is an esimation vector : dynamic array of (tag,value) pairs MA Contains standard performance metrics . . . LA 1 ? ? ? SeD response (DIET_profile_t) . . . t comp SeD 1 SeD 2 SeD 3 EST COMP_TIME COMM_TIME t comm m avail AVAIL_MEM Design and Implemention of a Plugin Scheduler for Diet & Perfo

Background on Diet Plugin Scheduler Design CoRI Implementation Conclusion & Future Works Plugin Scheduling Enhancements Example: Client request for comparison operation on blue database or Juxmem repository Request arrives at SeD level Diet hierarchy: Profile is an esimation vector : dynamic array of (tag,value) pairs MA Contains standard performance metrics and application-specific . . . LA 1 data ? ? ? SeD response (DIET_profile_t) . . . t comp SeD 1 SeD 2 SeD 3 EST COMP_TIME COMM_TIME t comm m avail AVAIL_MEM DB_BLUE Y DB_YELLOW N DB_PURPLE Y Design and Implemention of a Plugin Scheduler for Diet & Perfo