A Game-Theoretic Approach for Runtime Capacity Allocation in - PowerPoint PPT Presentation

A Game-Theoretic Approach for Runtime Capacity Allocation in MapReduce Eugenio Gianniti * , Danilo Ardagna * , Michele Ciavotta * , Mauro Passacantando * * * Politecnico di Milano ** Università di Pisa Slide 1 POLITECNICO DI MILANO

Goals Cost-effective deployment configuration for MapReduce systems hosted on private Clouds Distributed approach leveraging on local knowledge of problem parameters Efficient solution algorithm to support run-time cluster management Slide 2 POLITECNICO DI MILANO

Reference System M R M M Hadoop YARN R Slide 3 POLITECNICO DI MILANO

Preliminary Formulation N N X X min ρ r i + ¯ P i ( h i ) D ECISION V ARIABLES r , h , s M , s R h i — concurrency level i =1 i =1 r i — virtual machines subject to: s iM — Map slots s iR — Reduce slots N X r i ≤ R i =1 ≤ h i ≤ H up H low ∀ i ∈ A i , i A i h i + B i h i Integer Nonlinear Problem + E i ≤ 0 , ∀ i ∈ A s M s R i i s M + s R i i ≤ r i , ∀ i ∈ A Non-convex constraints c M c R i i r i ∈ N 0 , ∀ i ∈ A s M ∈ N 0 , ∀ i ∈ A Continuous relaxation i h i à ( 𝜔 i ) -1 s R i ∈ N 0 , ∀ i ∈ A h i ∈ N 0 , ∀ i ∈ A Slide 4 POLITECNICO DI MILANO

Centralized Problem N N D ECISION V ARIABLES X X min ρ r i + ¯ ( α i ψ i − β i ) r , ψ , s M , s R 𝜔 i — reciprocal concurrency level i =1 i =1 r i — virtual machines subject to: s iM — Map slots s iR — Reduce slots N X r i ≤ R i =1 ≤ ψ i ≤ ψ up ψ low i , ∀ i ∈ A i Continuous Nonlinear Problem A i B i + E i ≤ 0 , ∀ i ∈ A + s M s R i ψ i i ψ i Convex constraints s M + s R i i ≤ r i , ∀ i ∈ A c M c R i i KKT conditions are necessary and r i ∈ R + , ∀ i ∈ A sufficient for optimality s M ∈ R + , ∀ i ∈ A i s R i ∈ R + , ∀ i ∈ A Need to centralize information ψ i ∈ R + , ∀ i ∈ A Slide 5 POLITECNICO DI MILANO

Optimal Configuration Formulae P ROPOSITION Full knowledge of the problem parameters In the optimal configuration it holds: Exact bounds on resources c M s M i ∀ i ∈ A requirement = r i , i r c M B i 1 + i A i c R i c R ⌘ 2 s R i ⇣q q ∀ i ∈ A i = r i , A i B i + r c M c R c R r low A i i i = − , ∀ i ∈ A 1 + i E i ψ up i c M B i i i ⌘ 2 ⌘ 2 ⇣q ⇣q q q A i B i A i B i + + c M c R c M c R r up r − 1 i i i i ∀ i ∈ A ∀ i ∈ A ψ i = − , = − , i i E i ψ low E i i Slide 6 POLITECNICO DI MILANO

Application Masters Problems min α i ψ i − β i D ECISION V ARIABLES ψ i , ρ a i ,s M i ,s R i 𝜔 i — reciprocal concurrency level 𝜍 ia — bid for VMs subject to: s iM — Map slots i ≤ ρ up s iR — Reduce slots ρ ≤ ρ a ¯ i ≤ ψ i ≤ ψ up ψ low i i A i B i + E i ≤ 0 + s M s R i ψ i i ψ i Continuous Nonlinear Problem s M + s R i i ≤ r i c M c R i i One instance per AM s M ∈ R + i s R i ∈ R + Only application-specific ψ i ∈ R + information ρ a i ∈ R + Slide 7 POLITECNICO DI MILANO

Resource Manager Problem N N D ECISION V ARIABLES X X p i ( r up max ( ρ − ¯ ρ ) ˜ − r i ) r i − 𝜍 — price of VMs i r , y , ρ i =1 i =1 r i — virtual machines y i — AM i offers more than price subject to: N X r i ≤ R i =1 Mixed Integer Nonlinear Problem r i ≥ r low ∀ i ∈ A , i r up − r low y i + r low � � ∀ i ∈ A r i ≤ , i i i One instance for the whole cluster ρ ≤ ρ ≤ ˜ ¯ ρ ρ − ρ a i ≤ M (1 − y i ) , ∀ i ∈ A Takes care of resource ρ a ∀ i ∈ A i − ρ ≤ My i , management only ∀ i ∈ A r i ∈ R + , y i ∈ { 0 , 1 } , ∀ i ∈ A Two alternatives: r i = r i ˜ ρ ∈ R + and r i = r i − r low ˜ i Slide 8 POLITECNICO DI MILANO

Generalized Nash Equilibrium Problems A set of players, N , each with utility function Θ i Feasible set of player i : X i = X i ( x − i ) Solution concept: x ∈ X is equilibrium ⇔ ∀ i ∈ N, ∀ x i ∈ X i (¯ ¯ x − i ) , Θ i (¯ x ) ≤ Θ i ( x i , ¯ x − i ) GNEP JC-NEP Not jointly convex NEP Lack of theoretical guarantees Slide 9 POLITECNICO DI MILANO

Optimal Configuration Formulae P ROPOSITION The optimal configuration for every AM, given an amount of resources r i , is given by the following relations: c M s M i = r i i r c M B i 1 + i Local knowledge of application- A i c R i specific parameters c R s R i i = r i r c R A i Each AM problem reduces to a 1 + i c M B i i quick algebraic update ⌘ 2 ⇣q q A i B i + c M c R r − 1 i i ψ i = − i E i Slide 10 POLITECNICO DI MILANO

Best Reply Algorithm 1: r i ← r low , ∀ i ∈ A i ← s M, low 2: s M , ∀ i ∈ A i i i ← s R, low 3: s R , ∀ i ∈ A i 4: ψ i ← ψ up i , ∀ i ∈ A Each iteration is performed in 5: ρ a i ← ¯ ρ , ∀ i ∈ A parallel by the RM and AMs 6: repeat r old ← r i , ∀ i ∈ A 7: i solve RM problem 8: Continuous equilibrium for all i ∈ A do 9: configuration solve AM i problem 10: if ψ i > ψ low then 11: i i , ρ } + λρ up ρ a i ← max { ρ a 12: i end if 13: end for 14: | r i − r old i | ε ← P N 15: i =1 r old i 16: until ε < ¯ ε Slide 11 POLITECNICO DI MILANO

Experimental Overview P RELIMINARY A NALYSIS Ensure the model behavior is consistent with intuition when applied to simple problems S CALABILITY A NALYSIS Verify the feasibility of solving problem instances at a realistic scale in production environments S TOPPING C RITERION T OLERANCE A NALYSIS Check the sensitivity of the distributed algorithm with respect to the tolerance on the relative increment V ALIDATION WITH YARN SLS Compare model solutions and timings measured on the official simulator Slide 12 POLITECNICO DI MILANO

Preliminary Analysis 100 AMs 1,000 AMs Decreasing cluster capacity experiment, r i = r i − r low ˜ i Slide 13 POLITECNICO DI MILANO

Alternative Virtual Gain Terms r i = r i − r low ˜ ˜ r i = r i i Increasing concurrency level experiment, 10 AMs Slide 14 POLITECNICO DI MILANO

Scalability Analysis r i = r i − r low ˜ r i = r i ˜ i Execution time [s] Slide 15 POLITECNICO DI MILANO

Stopping Criterion Tolerance Analysis Relative error with respect to centralized solutions Slide 16 POLITECNICO DI MILANO

Validation with YARN SLS Users D i [s] S [s] η [%] Users D i [s] S [s] η [%] R R (10 , 15 , 20) 256 2740 2767.58 -0.996658 (20 , 10 , 10) 256 3090 2487.54 24.2191 (10 , 15 , 20) 512 1508 1447.12 4.20698 (20 , 10 , 10) 512 1694 1142.3 48.2977 (10 , 20 , 15) 256 2900 2708.3 7.07837 (20 , 10 , 20) 256 3380 2948.78 14.6237 (10 , 20 , 15) 512 1589 1379.58 15.18 (20 , 10 , 20) 512 1835 1299.59 41.1987 (15 , 10 , 10) 256 2575 2257.81 14.0487 (20 , 15 , 10) 256 3390 2849.85 18.9536 (15 , 10 , 10) 512 1456 980.087 48.5583 (20 , 15 , 10) 512 1845 1409.88 30.8625 (15 , 15 , 15) 256 3070 3183.02 -3.55061 (20 , 20 , 10) 256 3700 3339.28 10.8023 (15 , 15 , 15) 512 1678 1456.93 15.174 (20 , 20 , 10) 512 1995 1512 31.9444 (15 , 15 , 20) 256 3210 3088.88 3.92116 (20 , 20 , 15) 256 3845 3694.03 4.08677 (15 , 15 , 20) 512 1748 1547.92 12.9255 (20 , 20 , 15) 512 2063 1745.9 18.1626 (15 , 20 , 10) 256 3220 3070.87 4.85639 (20 , 20 , 20) 256 3987 4041.44 -1.34704 (15 , 20 , 10) 512 1755 1328.58 32.0956 (20 , 20 , 20) 512 2140 1877.4 13.9874 (15 , 20 , 20) 256 3512 3951.58 -11.1242 (25 , 15 , 25) 256 4300 3893.71 10.4346 (15 , 20 , 20) 512 1899 1574.1 20.6404 (25 , 15 , 25) 512 2290 1773.53 29.1213 (15 , 25 , 10) 256 3520 3276.66 7.42646 (25 , 25 , 25) 512 2597 2653.64 -2.13455 (15 , 25 , 10) 512 1906 1524.83 24.9975 Relative error absolute values average: 16.999 % Slide 17 POLITECNICO DI MILANO

Conclusions and Future Work The distributed approach yields accurate approximations of optimal solutions, even without theoretical guarantees Extend the formulation to Apache Tez and Spark (based on DAGs) Couple the proposed algorithm with a local search method based on Colored Petri Nets simulations Slide 18 POLITECNICO DI MILANO

Thanks for your attention… …any questions? Slide 19 POLITECNICO DI MILANO

Solution Rounding Algorithm 1: sort A according to increasing α i 2: r i d ˆ r i e , 8 i 2 A 3: for all j 2 A do if P N i =1 r i > R then 4: r j r j � 1 5: The RM runs an O(N) loop end if 6: 7: end for 8: s M ⌃ s M ⌥ , 8 i 2 A ˆ Each AM runs an O(1) loop, i i 9: s R ⌃ s R ⌥ i , 8 i 2 A ˆ concurrently i 10: for all j 2 A do while s M j /c M j + s R j /c R j > r j do 11: Sorting is O(N log N) , but can be s R j s R j � 1 12: done once and cached if s M j /c M j + s R j /c R j > r j then 13: s M s M j � 1 14: j end if 15: end while 16: 17: end for Slide 20 POLITECNICO DI MILANO

Preliminary Analysis 100 AMs 1,000 AMs Decreasing deadlines experiment, r i = r i − r low ˜ i Slide 21 POLITECNICO DI MILANO

Scalability Analysis r i = r i − r low ˜ ˜ r i = r i i Total cost and penalties Slide 22 POLITECNICO DI MILANO

Obtained Concurrency Levels Centralized model Closed form model Decreasing cluster capacity experiment Slide 23 POLITECNICO DI MILANO