! morning good CS 744: DRF Shivaram Venkataraman Fall 2020
ML knowledge ADMINISTRIVIA q TEY Attend tM%L - - - Assignment 2 out! fainted Piazza → , systems - Course Project → students - Form groups? Cloud ~ 3 Google Form Google - Project list by Monday (9/28) → - Submit project bids by Thursday (10/1) - Assigned project by Friday (10/2) work to 2 months on ~ project the
networking 5dm ! .us SETTING: FAIR SHARING z users lottery scheduling OS → earlier handle u different Equal Share Max-Min Share to → users demands across - Maximize the allocation for most poorly treated users Maximize the minimum
MOTIVATION: MULTI RESOURCES Memory Cpu Cpu manifested ÷÷:÷÷÷¥F÷÷
DRF: MODEL Users have a demand vector one task <2, 3, 1> means user’s task needs 2 R1, 3 R2, 1 R3 → - - - Resources given in multiples of demand vector demand their with i.e., users might get <4,6,2> tasks 2 ← - model based containers No shot are , " MMM " = isan : e groups g. war ; linux reduce 2 o o slots
PROPERTIES Sharing Incentive Strategy Proof about should get can't lie you User to get need what you 1am ? . ? - orison .at?east more - - - - - - . - Heir - truth off than Incentivize y worse No telling within cluster resources own Pareto Efficiency Envy free should allocate If envy not more users you need allocation , You another user for one of others to take away from user - - - . - - - - - - - - - - - - - utilization
PROPERTIES Sharing Incentive Strategy Proof User is no worse off than a User should not benefit by cluster with lying about demands 1/n resources Pareto Efficiency Envy free Not possible to increase User should not desire the one user without allocation of another user decreasing another
DRF: APPROACH Dominant Resource Dominant Share - - Resource user has the biggest Fraction of the dominant share of resource user is allocated Total: <10 CPU, 4 GB> E.g., for User 1 this is 25% or 1/4 User 1: <1 CPU, 1 GB> - - Dominant resource is memory Hare ÷ share men "
DRF: APPROACH Equalize the dominant share of users User Allocation Dominant Share <0 CPU, 0 GB> 0 2/9 Total: <9 CPU, 18 GB> Cl Cpu , 49137 User1 419 < 2. CPU , 8GB ? ← ICH , 129135J User1: <1 CPU, 4 GB> 61g - - dom res: mem - - I User2: <3 CPU, 1 GB> <0 CPU, 0 GB> 0 - dom res: CPU 53cm , 19137 3/9 - User2 , 29137T 61g k£00 used 14GB gcpu , Total : -
DRF: APPROACH Total: <9 CPU, 18 GB> User1: <1 CPU, 4 GB> per task o - <3 CPU, 12 GB> for 3 tasks - dom res: mem → tonnage D dom share: 12/18 = 2/3 - User2: <3 CPU, 1 GB> . is unallocated <6 GPU, 2 GB> for 2 tasks - dom res: CPU - O dom share: 6/9 = 2/3 -
DRF ALGORITHM Whenever there are available resources: Schedule a task to the user with smallest dominant share
DRF ALGORITHM config cluster → sum → running initialization → to given resources track → each user - ← < lcpu , 49137 = ] → if rn I is resource one fill still offer can you that tasks to resources need R don't ,
COMPARISON: ASSET FAIRNESS - Asset Fairness: Equalize each user’s sum of resource shares Violates Sharing Incentive Consider total of 70 CPUs, 70 GB RAM of resource units U1 needs <2 CPU, 2 GB RAM> per task 4 = resource of U2 needs <1 CPU, 2 GB RAM> per task units 3 ÷ = Asset Fair Allocation: u , for tasks ' : iii. iz 15 . ? :O U1: = 43 : ' tasks for us U2: w e . ,
COMPARISON: ASSET FAIRNESS Asset Fairness: Equalize each user’s sum of resource shares Violates Sharing Incentive u ' Ii 't . YI - be Hunter dedi at rat is Consider total of 70 CPUs, 70 GB RAM U1 needs <2 CPU, 2 GB RAM> per task U2 needs <1 CPU, 2 GB RAM> per task Asset Fair Allocation: U1: 15 tasks: 30 CPU, 30 GB (Sum = 60) = = U2: 20 tasks: 20 CPU, 40 GB (Sum = 60)
COMPARISON: CEEI CEEI: Competitive Equilibrium from Equal Incomes - Each user receives initially 1/n of every resource, - Subsequently, each user can trade resources with other users in a perfectly competitive market - Computed by maximizing product of utilities across users
dominant COMPARISON: CEEI resource Total: <9 CPU, 18 GB> User1: <1 CPU, 4 GB> User2: <3 CPU, 1 GB> - . CEE I f f Cpu - Mem 454 . - 62 l - 4.05 y = X - - , - - q t . ' ↳
CEEI: STRATEGY PROOFNESS be higher used to x. y Total: <9 CPU, 18 GB> 1.62 y = X ' 4.05 User2 Before: CEEI: 55% CPU, 9% mem tasks ? ? 3.6 ↳ discrete nursery tasks ? ! Total: <9 CPU, 18 GB> - → User1: <1 CPU, 4 GB> . f User2: <3 CPU, 2 GB> - Y X 9 nt3y I max , 3.6 18 fatty x g E - - I -8 y - -
COMPARISON
SUMMARY DRF: Dominant Resource Fairness Allocation policy for scheduling min fairness generalizes max - Provides multi-resource fairness → Ensures sharing incentive, strategy proofness
DISCUSSION https://forms.gle/i7m7xXxKhtfvL9UD9
Consider a system with 100 units of CPU, 50 units of memory and 200 units of disk. Consider three users with the following requirements Alice (4 CPU, 1 memory, 1 disk) - Z s Bob (1 CPU, 4 memory and 4 disk) Carol (1 CPU, 2 memory and 16 disk) List the dominant resource as defined in DRF for Alice, Bob and Carol 450 Alice CPO . 4150 Bob Memory : 4150 Disk Carol e .
What would be the final task allocation in the given cluster for Alice, Bob and Carol ? , card Bob Alice of tasks is Alice , nun X time y z Every , , " " . ! :* :* 8/200 onto allocated 16¥ 44 = 1640 44 . Card Bob = - - . 200 50 100 two turns i :* ; :* : : . .Y÷÷÷ get A- lice 1100 4kt y +2 . ' - . 12 Alice : 6 - Carol 6 , Bob 12-5 . XE - I or ( 12,6 , 7) ✓ 44,6 , b) 6.25--2 either yr .
What could be one workload / cluster scenario where DRF implemented on Mesos will NOT be optimal? enough aren't resources there If run task can → least one at if Instantaneously tasks Heterogeneous → . ? fair → ? ] time [ over ? pref Locality -71
Mesos : 18GB ) resource Offer ( 9900 , NEXT STEPS R : ( 21 , 21 > Ext at task IN , 318dB > → : Dal I. 17 Dl 22,27 : I tasks 3 Next Week: Machine Learning tasks 3 Zita Eisa Assignment 2 out! = tasks ) # - actor allocation rational proof assuming strategy Dpr : . :÷÷f÷÷÷÷ 3121 long running starvation :-) D1 : DRF one contended cluster highly 4M pz & : very 6/21 DI ' r F me . . . . ÷÷ on are . - IIe - ? ? constraints subject to resource
Recommend
More recommend