The Demand of Bulk Transfers over WAN More demanding 1.Transfer - PowerPoint PPT Presentation

The Demand of Bulk Transfers over WAN More demanding 1.Transfer large size 2.Minimize completion time More willing to 1.Provide demand information 2.Control its transfers 1

Software-Defined Networking (SDN) in WAN Global traffic engineering with centralized control, e.g., Google B4, Microsoft SWAN Given Traffic Demand Controller Network Topology Compute Traffic Engineering Central Control Routing Daily Timescales Minutes Rate Allocation Optimize Completion Time Deadlines Met 2

Network Layer over Optical Layer Seattle Network Network Layer Link Manual Config. Automated Config. Los Angeles Network switch (Router) Monthly Timescales Minutes Seattle Optical Optical Layer Circuit Los Angeles Optical switch 3

Technology Trends • Bulk-transfer applications with demand information • Fast centralized control with SDN • Fast reconfigurable optics 4

Reconfigure Optical Layer to Change Network-Layer Topology Configuration A 10 R 1 R 0 Network Layer 10 10 R 2 R 3 10 R 0 O 0 O 1 R 1 Optical Layer O 3 R 2 O 2 R 3 Router Optical Switch 5

Reconfigure Optical Layer to Change Network-Layer Topology Configuration A Configuration B 10 10 R 1 R 0 R 1 R 0 10 10 10 Network Layer 10 R 2 R 3 R 2 R 3 10 10 R 0 O 0 O 1 R 1 R 0 R 1 O 0 O 1 Optical Layer O 3 O 3 R 2 O 2 R 2 O 2 R 3 R 3 Router Optical Switch 6

Reduce Average Transfer Completion Time Routing + Rate Routing Routing + Rate allocation + Topology Step 2 Step 1 F 0 (Demand=10) F 0 (Demand=10) F 0 (Demand=10) 10 R 1 R 0 R 1 R 1 R 0 R 0 R 1 R 0 10 10 10 10 10 10 10 10 10 10 10 10 10 10 R 2 R 3 R 2 R 3 R 2 R 3 R 2 R 3 10 F 1 (Demand=10) F 1 (Demand=10) F 1 (Demand=10) Inefficiently Used Unused Capacity Capacity 7

Joint Optimization and Challenges 8

Joint Optimization Router Given Traffic Demand Optical Switch Client DC Constraints CurrentTopology C 1 R 1 Optical I nfrastructure Optical I nfrastructure # of Router Ports # of Router Ports R 0 C 0 O 1 Compute NewTopology NewTopology Optical Reach Optical Reach O 0 C 2 R 2 Routing # of Regenerators Rate Allocation O 2 # of Wavelengths Optimize Completion Time Wide Area Network Link Capacity Deadlines Met 9

Challenges • Efficient joint optimization • Routing • Rate allocation • Topology • Transition gracefully • Minimize disruption during update 10

Finding Good Configuration with Small Change Good Close Throughput Current Configuration Space 11

Simulated Annealing Algorithm Choose Random Neighbor Evaluate Neighbor Current 12

Owan's Solution Overview • Joint optimization efficiently Random Neighbor Topo. Choose • Avoids disruption Random Neighbor Optimize Network Layer Evaluate Neighbor Consistent Update 13

Owan Algorithm 14

Random Neighbor Topology 1. Make random local change Random Neighbor Topo. 2. Select optical circuits Optimize Network Layer Evaluate Neighbor Consistent Update 15

Random Neighbor Topology • Make random local change • Minimize changes to the network • Satisfy the port number constraints • Select optical circuits for each link 10+10 20 10 R 0 R 1 R 0 R 1 R 0 R 1 • Use graph algorithm • For each path 10-10 10 10 10-10 • Minimize regenerator usage R 2 R 3 R 2 R 3 R 2 R 3 • Balance regenerator usage across sites 10 10+10 20 Current Topology Random Local New Topology Change 16

Optimize Network Layer Random Neighbor Topo. 1. Routing Optimize Network Layer 2. Rate allocation Evaluate Neighbor Consistent Update 17

Schedule Transfers on the New Topology • Order transfers with classic scheduling disciplines SJF EDF …… Avg.Transfer # of Deadlines Met Other Objective CompletionTime • Prioritize short paths in rate allocation 18

Evaluate Neighbor Topology Random Neighbor Topo. Optimize Network Layer • Throughput: sum of rates Evaluate Neighbor Consistent Update 19

Consistent Update Random Neighbor Topo. Optimize Network Layer Evaluate Neighbor • Dependencies of operations Consistent Update 20

Implementation and Evaluation 21

Testbed Implementation ROADM • 9 Sites Arista • Emulating Internet 2 network One Switch • 135 servers Site Servers • Two 6-core Intel E5-2620v2 • 10GE 22

Evaluation • Workload • Generate transfers for 2 hours • Draw transfer size from exponential distribution • Mean 500GB/5TB for testbed/simulation • Evaluation • Testbed experiments, with 9 sites • Large-scale simulations, with about 40 sites • Results • Average transfer completion time: 3.5-4.4x • Number of transfers that meet deadlines: 1.1-1.3x 23

Deadline-Unconstrained Traffic • Performance metric • Transfer completion time • Other approaches • MaxFlow • MaxMinFract • SWAN[1] [1] Hong, Chi-Yao, et al., Achieving High Utilization with Software-Driven WAN, SIGCOMM 2013 24

Better Average Completion Time Better 4.45x 25

Deadline-Constrained Traffic • Performance metric • Percentage of transfers that meet deadlines • Amount of bytes that finish before deadlines • Other approaches • Deadline-unconstrained approaches • Amoeba[1] [1] Zhang, Hong, et al., Guaranteeing deadlines for inter-datacenter transfers, EuroSys 2015 26

More Transfers Meet Deadlines 1.36x Better 27

Consistent Update Avoids Disruptions Topology Update 28

Conclusions • Optical control improves WAN performance • Efficient algorithms for joint optimization • Transition gracefully 29

Thanks! Q&A

Build Optical Circuits for Each Link • Build regenerator graph • Balance regenerator consumption Goal: Find path with 0.2 0.25 min total node weight O 1 O 2 0 0 O 0 O 4 Shortest path problem 1 on directed graph O 3 Distance <= Optical Reach Inverse of # Regenerators 31

Cross-Layer Optimization at Each Time Slot Router Optical Switch Controller Client DC Request Submission Rate Allocation Topology Routing R 1 C 1 O 1 R 0 C 0 O 0 C 2 R 2 O 2 Wide Area Network 32