MIND: Machine Learning based Network Dynamics Dr. Yanhui Geng - PowerPoint PPT Presentation

MIND: Machine Learning based Network Dynamics Dr. Yanhui Geng Huawei Noah’s Ark Lab, Hong Kong

Outline • Challenges with traditional SDN • MIND architecture • Experiment results • Conclusion

Challenges with Traditional SDN Challenges from network scale and traffic volume Node # : 1000  100,000  100,000,000 (IoT)  Traffic (flow/s) : 10,000  1,000,000  1,000,000,000  Type of application : 10  100  1,000 (multi-tenant)  Constrains : 10  100  1,000 (dynamic)  Control Learning Foundation Causal analysis (white box) Data association (black box) Output Deterministic Stochastic Better interpretability, good Balance between optimality and Advantage performance under consistent exploration, self-learning and evolving, scenarios better scalability and robustness Difficult to adapt when there are Policies may be not optimal (poor ones Limitation changes with scenarios or with small probability), weak applications, poor scalability interpretability

Challenges with Traditional SDN SDN future  Dynamic control automation SDN now  Application awareness and adaption  OpenFlow protocol  Real-time policy generation  Functionality decouple  Better scalability and robustness  Centralized control  Richer application interface  The evolution from connectivity to intelligence will be driven by introducing machine learning and big data analytics into traditional network control and optimization!

Outline • Challenges with traditional SDN • MIND architecture • Experiment results • Conclusion

MIND architecture: automated control strategy generation enabled by machine learning Technical foundation : reinforcement learning + data mining + optimization

Least Congested Routing based on Elephant Flow Prediction Performance: Performance: TPR: 0.9787 99-th percentile of the average normalized elephant flow (percentage of elephant completion time improves over flows correctly predicted ) 10% compared to ECMP. FPR: 0.0577 (percentage of mice flows mistakenly predicted) Highlight 2: Online Learning Key Contributions ：  Propose flow size prediction based on packet header.  Employ Gaussian Process Regression to train the prediction model. Π T Π T k k k k          Develop an online GPR algorithm. α α Π n n 1 n 1 n n 1 n 1 ( I ) y ( I ) k    1  1  1 n Π n n Π n n  Develop Least Congested Routing Algorithm based on Elephant Flow Prediction. T T 1 1 k k k k     n 1 n n 1 n 1 n n 1 Highlight 1: Flow size prediction based on pattern similarities Highlight 3: Least Congested Routing based on Elephant Flow Prediction | , , ~ ( , cov( )) f X y x N f f          2 1 f E [ f | X , y , x ] K ( x , X ) [ K ( X , X ) I ] y  We use our develop algorithm to predict elephant flows              n and then route the predicted elephant flow to the least α      congested path and use ECMP to route mice flows 2 1 cov( ) ( , ) ( , )[ ( , ) ] ( , ) f K x x K x X K X X I K X x      n   α T ( ) ( ) f x k x   SR m n 1

Online Coflow Identification Fig1. Accuracy : three level features Fig2. Accuracy: without community Fig3. Accuracy: without weight and weight matrix detection matrix Community level Key Contributions ： Highlight 2: Weight matrix  Develop a machine learning based method to identify coflows in network.  Investigate features in three different levels.  Develop a learning based method to determine weights for different features.   T ( , ) ( , ) ( , ) d D i j D i j A D i j Highlight 1: Features in three levels ij A Application level   Flow level     * ( , ) arg min log( ) D i j A d d time     0 A ij ij if i and j Cmnty host 1 host 2 host 3 ( , )  D i j      ( , ) ( , ) ( , ) D com i j f f S f f D  size i j i j D ( i , j )    1 otherwise flow     learn optimal A from training data  ( , )  D i j int   0 if i and j AGG   D app ( i , j )   1 J ( i , j ) otherwise host 4 host 5 host 6

Reinforcement Learning in SDNs for Routing         ( , ) ( , ) [ ( , ) max ( , ) ( , )] Q s a Q s a R s a Q s a Q s a  a   ( s ) arg max Q ( s , a ) a Idea ：  We approximate the Q-function with a neural network . Advantages  We update the neural network using stochastic gradient descent. Learn changing traffic patterns:  Deep architectures can compactly represent functions that may need a Adaptive approach. very large shallow architecture. Tailored to the needs of network operator or  We argue that SDNs are characterized by an inherent locality that we applications: can exploit when designing a deep architecture with local connectivity All you need to do is define proper reward. patterns .  Options : stochastic RMSProp, prioritized experience replay, target network optimizations, double Q-learning.

Routing by Online-REPS-RKHS policy Observations: prob. The policy keeps improving as reward keeps increasing and FCT keeps decreasing. p2  p1 path Key Contributions: Highlight 1: Online-REPS-RKHS algorithm  Develop an online direct policy search algorithm based on the state-of-the art REPS-RKHS.      a max ( ) max ( | ) ( ) J a s s R dsda      , , s  Investigation on the network control problem using Reinforcement learning.    S A  Apply the Online-RKHS-REPS to network routing control.      s . t . ( a | s ) ( s ) dsda 1   S A        ( a , ) P ( s | s , a ) ( s | a ) ( s ) dsda ( s ) q s   Highlight 2: Online-REPS-RKHS algorithm to Network Routing Control  S A     ( ( | ) ( ) || ( , )) KL s a s q s a   We use Online-REPS-RKHS algorithm to learn the probability Data Block Sample 1 Sample 2 distribution of choosing the top-k best path.     0 1 2

Outline • Challenges with traditional SDN • MIND architecture • Experiment results • Conclusion

Experiments on test bed: configuration E F 172.16.2.2 172.16.1.2 172.16.7.2 172.16.8.2 Scenarios : DC with 800 virtual computation nodes,  172.16.6.2 spine-leaf topology (40+80 SDN switches), 10G 172.16.5.2 172.16.4.2 172.16.3.2 Ethernet connection Application/Traffic : real trace from Jiangsu Telecom  and Facebook, 40% MR job, 30% storage, 20% short 172.16.1.1 172.16.2.1 172.16.3.1 172.16.4.1 172.16.5.1 172.16.6.1 172.16.7.1 172.16.8.1 packets, 10% web query Performance metric : average job transmission time.  A B C D m10 m19 m1 m28 m11 m20 m2 m29 ... ... ... ... m18 m27 m36 m9 Rack&A Rack&B Rack&C Rack&D

Experiments on test bed: results Average job transmission time 60 50 40 30 20 ECMP 10 Hedera MIND2 0 time 1 2 3 4 5 6 7 8 9 10

Outline • Challenges with traditional SDN • MIND architecture • Experiment results • Conclusion

Conclusion • Propose and implement data-driven network control architecture MIND; • Online machine learning techniques for prediction/inference the spatial- temporal traffic information; • Reinforcement learning for optimal routing strategy based on traffic data and network state; • Experiment results with a real SDN test bed demonstrate the feasibility and effectiveness of the self-learning SDN paradigm;

Thank you!