Non-preemptive Coflow Scheduling and Routing Ruozhou Yu , Guoliang - PowerPoint PPT Presentation

IEEE Globecom 2016 SAC-ANS 3 Non-preemptive Coflow Scheduling and Routing Ruozhou Yu , Guoliang Xue, and Xiang Zhang Arizona State University Jian Tang Syracuse University 1/22

Outline q Introduction and Motivation q System Model and Algorithm Design q Performance Evaluation q Conclusion 2/22

Flows and Coflows Cloud Task q OMG where’s my last piece??? Need it now!!! Data piece 6 Data piece 1 Data piece 2 Data piece 5 Data piece 3 Data piece 4 3/22

Flows and Coflows q Traditional network scheduling/routing solution v Scheduling/Routing regarding individual flows v General flow: a subset of packet header fields v Fails to account for application-level performance metrics v Flow completion time vs. task completion time 4/22

Flows and Coflows Task 2 Task 1 You faster, you go ahead J Oh you’re so nice!! 5/22

Flows and Coflows q Application-aware scheduling/routing: coflows v Flows grouped by application/task v A coflow finishes when all its component flows finish v Advantages: v Captures application-level requirement v Establishes fairness in application-level v Want to do it in a centralized way v Not to leak app privacy to other apps v Or to prevent apps from selfishly congest the network 6/22

(Non-)Preemptive Scheduling q Existing coflow scheduling/routing allows preemption ! v Pause for the shorter ones! v Advantages: v Better performance and network utilization in theory v Disadvantages: v Large overhead for flow switching: performance issue for short flows q Switching delays q Switch computations v No ready support in commodity hardware q Standardization on-going: IEEE 802.1Qbu q A long way before commercial-ready q Our stand: non-preemptive scheduling + routing of coflows 7/22

Summary of Problem Task 1 Task 2 Now, you go first, this way! You next, that way! BOSS Sorry, there’s no place. You fired! You this way, free to go! Task 4 Task 3 8/22

Contributions q A first (preliminary) study for Non-preemptive Coflow Scheduling and Routing (NCSR) q An offline scheduling framework: Shortest-Coflow First q A multi-path routing algorithm q A single-path routing algorithm q Performance evaluations 9/22

Outline q Introduction and Motivation q System Model and Algorithm Design q Performance Evaluation q Conclusion 10/22

System Model q Network: G = ( V , E ) q Coflow requests: S = { C 1 , …, C m } v Each request: C i = { F i , 1 , …, F i , ni } v F i , j = ( s i , j , t i , j , d i , j ) : source, destination, flow size (demand, in bytes) q Bandwidth allocation v B p i , j ( t ) : bandwidth allocation on path p of flow i , j , at time t v B i , j ( t ) = sum of bandwidth over all paths at time t 11/22

System Model q Flow/coflow completion time v Flow completion time (FCT): v Coflow completion time (CCT): max. FCT of its component flows v Objective: minimize total CCT 12/22

Shortest-Coflow First Scheduler q For each coflow: v Compute per-coflow completion time (CCT) v If multi-path enabled, compute using multi-path routing v Otherwise, use single-path routing q Schedule coflows in ascending order of CCT 13/22

CCT with Multi-path Routing q Non-linear programming formulation v Sharing among flows within the coflow v CCT as the maximum FCT of component flows q Linearization: let f i = 1 / T i 14/22

CCT with Single-path Routing q Additional integer variables to the Multi-path Routing model v x e i , j : link selection for single-path routing q Linear relaxation and deterministic rounding v Relax x e i , j to take continuous values, and solve linear program; v For each flow, find path with maximum minimum x values, and assign; v Re-solve program to obtain bandwidth allocation with fixed path assignments 15/22

Outline q Introduction and Motivation q System Model and Algorithm Design q Performance Evaluation q Conclusion 16/22

Simulation Setups q Waxman random graphs v 50 nodes v Alpha=0.15, beta=0.2 v Link capacities: [10, 100] Mbps q Coflows v 25 requests v 1 to 10 flows per request v Flow sizes: [10, 100] Mbps q Comparison: v sSCF, mSCF: single-path and multi-path SCF algorithm (proposed) v sRT, mRT: single-path and multi-path Routing-only algorithm (baseline) v sSFF, mSFF: single-path and multi-path Shortest-Flow First (baseline) 17/22

Simulation Results: Average CCT 18/22

Simulation Results: Running Time 19/22

Outline q Introduction and Motivation q System Model and Algorithm Design q Performance Evaluation q Conclusion 20/22

Conclusions q A first step study on NCSR v Offline optimization model v SCF scheduler for scheduling v Multi-path and single-path routing algorithms q Experiment results v Scheduling more effective than routing: when network congested v Application-awareness brings great advantage q Future work v Enable better sharing/work conservation of resources v Remove the non-sharing rule of coflows 21/22

Q&A? THANK YOU VERY MUCH! 22/22