An Advance Reservation ‐ based Co ‐ allocation Algorithm for Distributed Computers and Network Bandwidth on QoS ‐ guaranteed Grids Atsuko Takefusa, Hidemoto Nakada, Tomohiro Kudoh, and Yoshio Tanaka National Institute of Advanced Industrial Science and Technology (AIST)
Resource Co ‐ allocation for QoS ‐ Q guaranteed Grids • QoS is a key issue on Grid/Cloud – Network (=Internet) is shared by abundant users • Network resource management technologies have enabled the construction of QoS ‐ guaranteed Grids – Dynamic resource co ‐ allocation demonstrated G ‐ lambda and EnLIGHTened Computing [GLIF06,SC06] – Each network is dedicated and dynamically d d ll provisioned – No connectivity N ti it R Resource co ‐ allocation is ll ti i w/o reservation an important technology 2
Preconditions of Our Co ‐ allocation Preconditions of Our Co allocation • Commercial services – Some resources including network are provided by resource managers (RM) from commercial sectors – The resources will be charged h ll b h d – The RMs do not disclose all of resource information • Advance reservation – Prediction ‐ based scheduling systems, e.g. KOALA and QBETS, cannot guaranteed to activate co ‐ allocated resources at the same time • The user has to pay for some commercial resources during the Th h t f i l d i th waiting time • On ‐ line reservation service • On line reservation service – Try to complete resource co ‐ allocation, quickly 3
Issues for Resource Co ‐ allocation for QoS ‐ guaranteed Grids (1/2) • Co ‐ allocation of both computing and network resources – There are constraints between computers and the network links – Cannot use list scheduling ‐ based approaches and network routing algorithms based on Dijkstra's algorithm, straightforwardly t i htf dl • Reflecting scheduling options – Users: (a) reservation time, (b) price, and (c) quality/availability – Administrators: (A) load balancing among RMs, d i i ( ) l d b l i (B) preference allocation, and (C) user priority 4
Issues for Resource Co ‐ allocation for QoS ‐ guaranteed Grids (2/2) • Calculation time of resource co ‐ allocation – Resource scheduling problems are known as NP ‐ hard – Important to determine co ‐ allocation plans with short calculation time, especially for on ‐ line services 5
Our Contribution Our Contribution • Propose an on line advance reservation based co • Propose an on ‐ line advance reservation ‐ based co ‐ allocation algorithm for distributed computers and network bandwidths network bandwidths – Model this resource co ‐ allocation problem as an integer programming (IP) problem programming (IP) problem – Enable to apply the user and administrator options • Evaluate the algorithm with extensive simulation, in • Evaluate the algorithm with extensive simulation in terms of functionality and practicality – Can co ‐ allocate both resources and can take the C ll t b th d t k th administrator options as a first step – Planning times using a general IP solver are acceptable for Planning times using a general IP solver are acceptable for an on ‐ line service 6
The Rest of the Talk The Rest of the Talk • Our on line advance reservation based co allocation • Our on ‐ line advance reservation ‐ based co ‐ allocation model • An advance reservation ‐ based co ‐ allocation algorithm A d ti b d ll ti l ith – Modeled as an IP problem • Experiments on functional and practical issues • Related work • Conclusions and future work 7
Our On ‐ line Advance Reservation ‐ based Co ‐ allocation Model • Consists of Global Resource Coordinators (GRCs) f l b l d ( ) (= Grid scheduler) and resource managers (RMs) • Each RM manages its User / Application reservation timetable and discloses a part of the Grid Resource Planning resource information Coordinator (GRC) • GRC creates GRC Network Compute RM RM reservation plans p CRM CRM NRM NRM NRM NRM and allocates the resources CRM CRM SRM SRM CRM CRM CRM CRM Network Network SRM SRM SRM SRM Network Network Domain A Domain B Storage RM 8
User Request and Reservation Plan User Request and Reservation Plan User Request Reservation Plan ? (Linux, Domain 2 5 SiteB MPI) 5 ? (LI ? (LInux, X1 X1 SiteC MPI) ? 1Gbps Abstract 1 10 1 10 (Linux, 1Gbps Network SiteA SiteA MPI) MPI) Domain 1 D i 1 Topology EarliestStartTime (EST) 1:00 StartTime (ST) 2:00 LatestStartTime (LST) 5:00 EndTime (ET) EndTime (ET) 3:00 3 00 Duration (D) 1 hour Resource Requirement Parameters • Compute resources: # of CPUs/Cores, Attributes (e.g., OS) • Network resources: Bandwidth, Latency, Attributes • Time frames: Exact (ST and ET) or range (EST, LST, and D) 9
The Steps of Resource Co allocation The Steps of Resource Co ‐ allocation 1 1. GRC GRC receives a co ‐ allocation request from User i ll i f U User / Application 2. GRC Planner creates reservation plans 2i. Selects N time frames from [EST, LST+D] 2i Selects N time frames from [EST LST+D] 1. Request 4. Result 2ii. Retrieves available resource information GRC at the N time frames from RMs 2iii. Determines N' ( N) co ‐ allocation plans iii i ' ( ) ll i l 2. Planning 2 Pl i using 2ii information Request Modeled as an IP problem Co ‐ allocator Planner 2iv. Sorts N' plans by suitable order Reservation 3. GRC tries to co ‐ allocate the selected Plans resources in coordination with the RMs resources in coordination with the RMs 4. GRC returns the co ‐ allocation result, 3. Resource 2ii. Retrieving Co-allocation available info whether it has succeeded or failed If failed, the User will resubmit an updated request Resource Managers (RMs) Resource Managers (RMs) Resource Managers (RMs) 10
Resource and Request Notations q Resources : G=(V, E) Request : G r =(V r , E r ) – v ( V ) : Compute resource site or v n ( V ) : Compute resource site or – vr ( V ) : Requested compute vr m ( V r ) : Requested compute network domain exchange point site – e o, p ( E ) : Path from v o to v p – er q, r ( E r ) : Requested network between vr q and vr r b d Resource parameters Request parameters – wc i ( i V ) : # of available CPUs wb ( k E ) : Available bandwidth wb k ( k E ) : Available bandwidth – rc j (j V r ) : Requested # of CPUs rc (j V ) : Requested # of CPUs ( e o, p and e p, o share the same wb k ) – rb l (l E r ) : Requested bandwidth – vc i ( i V ) : Value per unit of each CPU vr 1 5 5 ? ? vb k ( k E ) : Value per unit of bandwidth er 0, 1 er 1,2 v 0 v 4 ? 1 10 v 2 ? (10cpus) (20cpus) 1Gbps er 0, 2 e 0, 2 , e 2, 0 (1G) e 2, 4 , e 4, 2 (10G) vr 0 vr 2 e 2, 5 , e 5, 2 (10G) e 0, 3 , e 3, 0 (1G) e 0, 1 , e 4, 5 , e 1, 2 , e 2, 1 (10G) e 3, 4 , e 4, 3 (1G) e 1, 0 e 5, 4 (1G) (1G) (1G) (1G) e 1, 3 , e 3, 1 (1G) e 3, 5 , e 5, 3 (1G) v 1 v 5 Domain 1 Domain 2 v 3 (30cpus) (20cpus) 11
Modeling as a 0 1 IP Problem Modeling as a 0 ‐ 1 IP Problem x i, j {0, 1} (i V, j V r ) X (Compute site plan) X (Compute site plan) = {0 1} (i V j V ) (1) (1) Y (Network path plan) = y k, l {0, 1} (k=(m, n) E, m, n V, (2) l=(o, p) E r , o, p V r ) Request Domain 2 v 0 v 1 v 2 v 3 v 4 v 5 5 vr 1 =v 4 vr 0 0, 1, 0, 0, 0, 0 v 2 v 2 vr 2 =v 0 vr =v R Resources X = vr 1 0, 0, 0, 0, 1, 0 1Gbps 1 10 vr 2 1, 0, 0, 0, 0, 0 vr 0 =v 1 Domain 1 e 0,1 e 1,0 e 1,2 e 2,1 e 2,4 e 4,2 ... v 2 e 2 4 v 2 0,1 1,0 1,2 2,1 2,4 4,2 2,4 v 0 v 4 er 0, 1 0 0 1 0 1 0 ... Y = er 0, 2 0 1 0 0 0 0 ... e 1,0 e 1,2 er 1 2 e 1, 2 0 0 0 0 0 0 ... 0 0 0 0 0 0 v v 1 12
Objective function and Constraints Objective function and Constraints (3) : Minimize the sum of resource Minimize values l (3) vc rc x ve rb y i j j k l k , l i v , j v k E , l E r r (4),(5),(6) : Constraints on the ( ),( ),( ) compute site plan X Subject to (4) Select 1 site for each requested site ( ) (4) j V , x 1 r i , j i V (5) Each site is selected to 0/1 site (5) i V , x 1 i , j (6) Each selected site can provide j V r (6) i V , rc x wc j i , j i requested # of CPUs requested # of CPUs j j V V r 1 ( rb 0 ) (7),(8) : Constraints on the path plan Y l (7) l E , y r k , l 0 ( rb 0 ) l k E r (7) The sum of y k l becomes more than 1, ( ) y k,l , (8) (8) k k E E , rb rb y y wb wb l l k k , l l k k if requested l E r l ( o , p ) E , m V , r (8) Each selected path can guarantee x x ( rb 0 ) m , o m , p l (9) (9) y y y y ( ( n , m ) ), ( ( o , p p ) ) ( ( m , n ), ) ( ( o , p p ) ) requested bandwidth requested bandwidth 0 0 ( ( rb b 0 0 ) ) l n V , m n n V , m n (9) : Constraint on both X and Y 13
Recommend
More recommend
