Load Balancing for Interdependent IoT Microservices Ruozhou Yu, - PowerPoint PPT Presentation

Load Balancing for Interdependent IoT Microservices Ruozhou Yu, Vishnu Teja Kilari, Guoliang Xue Arizona State University Dejun Yang Colorado School of Mines

Outlines Background and Motivation System Modeling Algorithm Design and Analysis Performance Evaluation Discussions, Future Work and Conclusions 2

IoT: The Future Internet q IoT is the future Internet that connects every aspect of our work and life. Transportation Environment Home Agriculture Healthcare Shopping Security Manufacturing Travel 3

Example IoT Applications Analysis Traffic Object Traffic Image Temporal Detection Prediction Processing Analysis 4

Monolithic Applications Searching Criminal Face Object Face Image Recognition Detection Comparison Processing Analysis Traffic Object Traffic Image Temporal Detection Prediction Processing Analysis Detection Crowd Temporal Device Population Signal Analysis Matching Estimation Processing 5

Microservices Searching Criminal Face Face Recognition Comparison Image Object Detection Processing Analysis Traffic Traffic Prediction Temporal Analysis Detection Crowd Signal Device Population Processing Matching Estimation 6

Microservices vs. Edge Computing High delay High bw overhead Responsive Single point of failure Low bw overhead Security concerns Distributed Secure Cloud-based Edge-based 7

The Microservice Load Balancing Problem q Edge-based microservices can be easily saturated . Image Processing 8

The Microservice Load Balancing Problem q Edge-based microservices can be easily saturated . Image Image Processing Processing 1 Image Processing 2 9

The Microservice Load Balancing Problem q Edge-based microservices can be easily saturated . q Challenge: interdependent microservices. Object Detection 1 Image Temporal Processing Analysis 1 1 Object Traffic Detection Prediction 2 Image Temporal Processing Analysis 2 2 Object Detection 3 10

Our Approach: Overview Problem Modeling 1) DAG-based interdependency graph (App-Graph). 2) Compactly modeled infrastructure (Inf-Graph). 3) Flexible application instantiation (Real-Graph). 4) Joint instantiation finding & load allocation. 5) Application QoS requirements. Algorithmic Results 1) Optimal algorithm for QoS-agnostic problem. 2) NP-hardness for QoS-aware problem. 3) FPTAS for QoS-aware problem. Next Steps (Future Work) 1) Network-aware load balancing. 2) Reliability and security. 3) Economics-aware microservice composition. 11

Outlines Background and Motivation System Modeling Algorithm Design and Analysis Performance Evaluation Discussions, Future Work and Conclusions 12

Application with Interdependent Microservices q General DAG-based application graph (App-Graph). v Captures complex interdependencies, unlike existing line graph-based models. Data Input Data Input Data Input Microservice Microservice Microservice Microservice API Call API Call API Call Video Video Proc. Proc. Cloud Cloud Anomaly Anomaly Security Cameras Storage Storage Detect. Detect. Motion Motion Detect. Detect. Smart Smart Motion Sensors Cmd. Cmd. Home Home Center Center Control Control Data Data Proc. Proc. Interdependency edge Data distribution ratio: output / input. • Ambient Sensors Microservice 13

IoT Infrastructure in the Application’sView q Inf-Graph: deployed microservice instances & their interactions. App-Graph B 0.5 1.0 1.0 1.0 Inf-Graph A D E B 1 B 1 B 1 (1, 5) (1, 5) 0.5 1.0 (2, 3) (2, 3) (2, 4) (2, 4) C D 1 D 1 D 1 E 1 E 1 E 1 (2, 1) (2, 1) External Demand A 1 A 1 A 1 δ =2.0 δ =2.0 D 2 D 2 D 2 E 2 E 2 E 2 Microservice Instance Capacity: input data volume. • (2, 6) (2, 6) (2, 6) (2, 6) C 1 C 1 C 1 (1, 4) (1, 4) Processing delay . • Inter-Instance Communication Channel Transmission delay (=0 in example). • 14

Application Instantiation q Real-Graph: instantiating the App-Graph in the Inf-Graph. Selected Instance Inf-Graph • Impact ratio: how much data this instance B 1 B 1 B 1 B 1 B 1 B 1 (1, 5) (1, 5) (1, 5) (1, 5) (2, 3) (2, 3) (2, 3) (2, 3) (2, 4) (2, 4) (2, 4) (2, 4) gets if 1 unit is allocated to the source. • Max cumulative delay from source . D 1 D 1 D 1 D 1 D 1 D 1 E 1 E 1 E 1 E 1 E 1 E 1 (2, 1) (2, 1) (2, 1) (2, 1) • Both can be computed based on this graph. A 1 A 1 A 1 A 1 A 1 A 1 δ =2.0 δ =2.0 δ =2.0 δ =2.0 D 2 D 2 D 2 D 2 D 2 D 2 E 2 E 2 E 2 E 2 E 2 E 2 Real-Graph B 1 B 1 (2, 6) (2, 6) (2, 6) (2, 6) (2, 6) (2, 6) (2, 6) (2, 6) (0.5, 6) (0.5, 6) C 1 C 1 C 1 C 1 C 1 C 1 (1, 4) (1, 4) (1, 4) (1, 4) (1, 1) (1, 1) (1, 13) (1, 13) (0.5, 5) (0.5, 5) Source Demand Allocation φ =2.0 A 1 A 1 C 1 C 1 D 1 D 1 E 1 E 1 φ =2.0 (1, 9) (1, 9) (1, 7) (1, 7) (1, 13) (1, 13) Source of Demand D 2 D 2 E 2 E 2 • A real-graph has a single source of demand. (There could be multiple sources in Inf-Graph.) 15

Problem Statement: Overview q Inputs: v App-Graph: microservices, interdependencies, data distribution ratios v Inf-Graph: instances, communication channels, capacities, and delays q Outputs: v A set of Real-Graphs. v External demand allocation for each Real-Graph. q Constraints: v Load balancing : total load <= instance capacity * 𝛺 . v QoS awareness: maximum delay <= D . q Objective (optimization version): v Minimize maximum delay of all Real-Graphs. 16

Outlines Background and Motivation System Modeling Algorithm Design and Analysis Performance Evaluation Discussions, Future Work and Conclusions 17

<latexit sha1_base64="m9PQWmRwqT5p5Kb9x2EeaIVayvI=">ADQnicjVJLb9QwEHbCqyPbuHIZcQKtAsoyiIkXkKq4MKhQgti26L1NnIcp7Xq2MF2Cqso/40Lf4Abf4ALB0BcOeA8JNrtHhjJ8qeZ7/OMZybOBTc2DL96/pmz585fWLvYu3T5ytX1/sa1baMKTdmUKqH0bkwME1yqeVWsN1cM5LFgu3Ehy/q+M4R04Yr+dYucjbPyL7kKafEOle04b3Dln20ZcplUgHcxu8LkD6BLYAZyQ3VtW3PYj8k21dwcA416rMIEN/ilwoQlkYRnHdwrW5o7VeX8dwGbIotKAZhL2Iq6MJdVNZSjCtKhGN2D7rFUaSIEyIb76mTdTkRFgzwxHDANFEWaCT/R6jcnhUz+4nMeLXlWcKmxdXUteWaALNZLtkxKXYRE/UEYhI3BaTDuwAB1Non6X3CiaJExakgxszGYW7nJdGWU8GqHi4Mywk9JPts5qAkGTPzstmBCm45jxuc0u5I147ae1xRksyYRY7Zj1Psxyrnatis8Kmj+ZuTnlhmaRtorQ4DajXihIuGbUioUDhGruagV6QDSh1q1dzVhvPzl02B6P3gchK8fDafd91YQzfQTREY/QbaKXaIKmiHqfvG/eD+n/9n/7v/yf7dU3+s019EJ8/8BUIOB0A=</latexit> <latexit sha1_base64="m9PQWmRwqT5p5Kb9x2EeaIVayvI=">ADQnicjVJLb9QwEHbCqyPbuHIZcQKtAsoyiIkXkKq4MKhQgti26L1NnIcp7Xq2MF2Cqso/40Lf4Abf4ALB0BcOeA8JNrtHhjJ8qeZ7/OMZybOBTc2DL96/pmz585fWLvYu3T5ytX1/sa1baMKTdmUKqH0bkwME1yqeVWsN1cM5LFgu3Ehy/q+M4R04Yr+dYucjbPyL7kKafEOle04b3Dln20ZcplUgHcxu8LkD6BLYAZyQ3VtW3PYj8k21dwcA416rMIEN/ilwoQlkYRnHdwrW5o7VeX8dwGbIotKAZhL2Iq6MJdVNZSjCtKhGN2D7rFUaSIEyIb76mTdTkRFgzwxHDANFEWaCT/R6jcnhUz+4nMeLXlWcKmxdXUteWaALNZLtkxKXYRE/UEYhI3BaTDuwAB1Non6X3CiaJExakgxszGYW7nJdGWU8GqHi4Mywk9JPts5qAkGTPzstmBCm45jxuc0u5I147ae1xRksyYRY7Zj1Psxyrnatis8Kmj+ZuTnlhmaRtorQ4DajXihIuGbUioUDhGruagV6QDSh1q1dzVhvPzl02B6P3gchK8fDafd91YQzfQTREY/QbaKXaIKmiHqfvG/eD+n/9n/7v/yf7dU3+s019EJ8/8BUIOB0A=</latexit> <latexit sha1_base64="m9PQWmRwqT5p5Kb9x2EeaIVayvI=">ADQnicjVJLb9QwEHbCqyPbuHIZcQKtAsoyiIkXkKq4MKhQgti26L1NnIcp7Xq2MF2Cqso/40Lf4Abf4ALB0BcOeA8JNrtHhjJ8qeZ7/OMZybOBTc2DL96/pmz585fWLvYu3T5ytX1/sa1baMKTdmUKqH0bkwME1yqeVWsN1cM5LFgu3Ehy/q+M4R04Yr+dYucjbPyL7kKafEOle04b3Dln20ZcplUgHcxu8LkD6BLYAZyQ3VtW3PYj8k21dwcA416rMIEN/ilwoQlkYRnHdwrW5o7VeX8dwGbIotKAZhL2Iq6MJdVNZSjCtKhGN2D7rFUaSIEyIb76mTdTkRFgzwxHDANFEWaCT/R6jcnhUz+4nMeLXlWcKmxdXUteWaALNZLtkxKXYRE/UEYhI3BaTDuwAB1Non6X3CiaJExakgxszGYW7nJdGWU8GqHi4Mywk9JPts5qAkGTPzstmBCm45jxuc0u5I147ae1xRksyYRY7Zj1Psxyrnatis8Kmj+ZuTnlhmaRtorQ4DajXihIuGbUioUDhGruagV6QDSh1q1dzVhvPzl02B6P3gchK8fDafd91YQzfQTREY/QbaKXaIKmiHqfvG/eD+n/9n/7v/yf7dU3+s019EJ8/8BUIOB0A=</latexit> BLB: Basic (QoS-agnostic) Load Balancing q Without delay constraint, the problem can be formulated as LP. f : L 7! R ∗ Variables: Per-link demand allocation function. find δ n = δ ext X s.t. + f ( l ) , 8 n 2 N ; n l ∈ L in ( n ) Demand per node = external + flow-in. 8 n 2 N ; δ n  Ψ · c n , X r ( v n ,w ) δ n = f ( l ) , 8 n, w 2 V out ( v n ) . l ∈ L out ( n,w ) Capacity (load balancing) per node. Flow conservation: sum flow towards all instances of a downstream microservice w = input data * data distribution ratio of w . 18