EdgeBalance: Model-Based Load Balancing for Network Edge Data Planes Wei Zhang † , Abhigyan Sharma * , Timothy Wood † † George Washington University * AT&T Labs Research 1
Outline • Background and Motivation • EdgeBalance Design • Environment Setup and Evaluation • Conclusion • Wei Zhang 2
Network Edge Computing • Current User Plane Control Plane Wei Zhang 3
Network Edge Computing • Current User Plane • 5G Edge Control Plane User Plane User Plane Control Plane User Plane User Plane Wei Zhang 4
Network Edge Computing • Current User Plane • 5G Edge Control Plane User Plane User Plane Control Plane User Plane User Plane User plane more distributed More network services can be pushed into edge Wei Zhang 5
Edge Computing runs NFs • Network services comprise of chains Wei Zhang 6
Edge Computing runs NFs • Replicated NF chains require a load balancer Wei Zhang 7
Cloud Load Balancer • Homogeneous services, static policies Web • Google Maglev • Microsoft Ananta Web • Facebook Mcrouter Web • Twitter TwemProxy • …… Memcached Memcached Memcached Memcached Wei Zhang 8
Cloud Load Balancer • Homogeneous services, static policies Web • Google Maglev • Microsoft Ananta Web • Facebook Mcrouter Web • Twitter TwemProxy LB for NF Chains • …… VPN Snort FW Memcached Memcached VPN Snort FW Memcached Memcached VPN Snort FW Wei Zhang 9
Bidirectional Affinity • Forward and reverse flows are required to pass through the same set of network functions Update header NF NF NF NF Client Server Load Balancer Client Server Wei Zhang 10
Static LB is not enough • Dynamic server resources • Dynamic service chains 100 • Dynamic workloads 75 Processing Cost NF1 NF2 NF3 50 NF4 NF5 NF6 25 NF1 NF6 0 NF1 NF2 NF3 NF4 NF5 NF6 Different Processing Cost of NFs Different Length of NF Chains Wei Zhang 11
Challenges for Dynamic LB • Fine-grained metrics are required for dynamic load balancing • Real time CPU monitoring is • Impossible for poll mode NFs 100 • May have stale data 75 • Increases system overhead CPU Usage 50 25 0 0 20 40 60 80 100 120 Wei Zhang 12
Outline • Background and Motivation • EdgeBalance Design • Environment Setup and Evaluation • Conclusion • Wei Zhang 13
EdgeBalance 1. Bidirectional affinity 3. Dynamic load balancing 2. Accurate CPU prediction 4. High performance Client Chain2 Chain1 NF NF NF NF Rule Client C1 C2 Host1 Host 1, Core 2 Chain 2 Host 2, Core 2 Chain2 Chain1 Flow 1 (purple) Host 1, Core 2, Chain2 Server NF NF NF NF Flow 2 (yellow) Host 2, Core 2, Chain2 Flow Table Load Balancer C1 C2 Host2 Server Wei Zhang 14
Bidirectional Affinity • Add forward flow and reverse flow into forward table and reverse table Update dst port NF11 NF21 NF31 Flow1 NF12 NF22 NF32 Flow2 5 Tuple NF Chains <srcIP1, dstIP1, udp, srcPort1, dstPort1> NF11->NF21->NF31 <srcIP2, dstIP2, tcp, srcPort2, dstPort2> NF12->NF22->NF32 Forward Table 5 Tuple NF Chains <dstIP1, srcIP1, udp, dstPort11, srcPort1> NF11->NF21->NF31 <dstIP2, srcIP2, tcp, dstPort2, srcPort2> NF12->NF22->NF32 Reverse Table Wei Zhang 15
Bidirectional Affinity • Remove forward flow and reverse flow from forward table and reverse table NF11 NF21 NF31 Flow1 NF12 NF22 NF32 5 Tuple NF Chains <srcIP1, dstIP1, udp, srcPort1, dstPort1> NF11->NF21->NF31 Forward Table 5 Tuple NF Chains <dstIP1, srcIP1, udp, dstPort11, srcPort1> NF11->NF21->NF31 Reverse Table Wei Zhang 16
Accurate CPU Prediction • Model to predict CPU usage NF7 NF8 NF9 • Chain processing cost NF4 NF5 NF6 • The number of flows NF1 NF3 NF2 • Flow rate Core Wei Zhang 17
Accurate CPU Prediction • Model to predict CPU usage NF7 NF8 NF9 • Chain processing cost NF4 NF5 NF6 • The number of flows NF1 NF3 NF2 • Flow rate Core <hostID, coreID, chainID> Stats (bits, packets) <host1, core1, chain1> xx, yy n ∑ Pr edict _ CPU ijk = Cos t ijk * Rate ijk k = 1 i : host I j : core j k : number of chains Wei Zhang 18
Dynamic Load Balancing • Balance the load across the hosts and cores • PID controller for LB weight setting - Seeks to equalize load on hosts and cores - Uses local information for feedback … C1 CM Disturbance Diff_PCPU_i Host1 Average_PCPU Weight_i Target system: PCPU_i Controller Weight? Evenly CPU usage - … C1 CN Hostk Wei Zhang 19
High Performance • DPDK high network IO User Space Flow Table Flow Table • Multi-threads - RSS dispatch flows to cores • Lockless flow table Core1 Core1 - Per-core flow table Kernel - Cleanup on the same thread Space NIC RSS Wei Zhang 20
Outline • Background and Motivation • EdgeBalance Design • Environment Setup and Evaluation • Conclusion • Wei Zhang 21
Experiment Setup • Cloudlab c220g5 specs Pktgen-DPDK Cisco-Trex • DPDK 18.02 • Ubuntu 14.04 EdgeBalance • Packet generator - Pktgen-DPDK NF2 NF3 NF1 NF2 NF3 NF1 - Cisco Trex • Bess runs NFs Bess Bess Wei Zhang 22
Evaluation: Prediction Accuracy • Accuracy of model-based CPU use prediction 100 Measured CPU 90 Predicted CPU 80 CPU Usage (%) 70 60 50 40 30 20 10 0 180 300 700 1400 1850 Processing cost of NF chain (ns) Wei Zhang 23
Evaluation: Outperform • A workload with a mix of elephant and mice flows • EdgeBalance outperforms static and monitor-based LBs 400 1600 Static WRR Monitor 350 1400 EdgeBalace Throughput (Mbps) 300 1200 Latency (us) 250 1000 200 800 150 600 100 400 Static WRR 50 200 Monitor EdgeBalace 0 0 0 5 10 15 20 25 30 35 40 45 50 0 5 10 15 20 25 30 35 40 45 50 Elephant traf f c ratio (%) Elephant traf f c ratio (%) Wei Zhang 24
Conclusions • EdgeBalance aims to balance the load for network service chains at edge • Network edge has unique challenges ➡ Limited resources ➡ Heterogeniety ➡ Affinity due to stateful network functions • Bidirectional flow affinity ➡ Forward flow and reverse flow go through the same set of NFs • Dynamic load balance ➡ Accurately predict CPU load ➡ Dynamically adjust allocation weight with PID controller to balance CPU load across hosts wei.zhang.gb@gmail.com Wei Zhang 25
Recommend
More recommend
