Operator Placement for Stream-Processing Systems Written by Peter - PowerPoint PPT Presentation

Network-Aware Operator Placement for Stream-Processing Systems Written by Peter Pietzuch, Jonathan Ledlie, Jeffrey Shneidman, Mema Roussopoulos, Matt Welsh, Margo Seltzer Presented by Ee Lee Ng February 11 Slides adapted from ICDE06 ppt

Large-Scale Stream-Processing Many geographically distributed data sources  e.g., sensors, network routers, RFID tag readers, … • High volume of real-time stream data Many users, submitting individual stream queries • Queries use the Internet for stream transport Queries include operators for stream-processing  e.g., join, filter, aggregate, XPath, image analysis, … • Operators require nodes for execution • In-network processing can often reduce data volume 2

Stream-Based Overlay Network transform aggregate SBON Node Data Source User 3

Operator Placement Problem How do you map operators to overlay nodes? Efficiency • Node and network resources are limited and shared • Operator placement must be network-aware  Consider link latency, bandwidth, congestion, jitter, …  Filter and aggregate data close to sources Scalability • Must scale to many sources, overlay nodes and queries • No global view of the system Adaptability • Resource conditions change over time 4

Contributions Stream-Based Overlay Network (SBON) • Generic layer between network and stream-processing apps • Shields applications from network complexity Operator placement using a metric cost space • Decentralized framework for minimizing network impact • Relaxation placement algorithm for operator placement • Adaptive to change in network conditions Deployment of SBON and sample applications (Borealis extension) on PlanetLab 5

Operator Placement Goals Two conflicting optimization goals: 1. Global system performance with concurrent queries  Minimize network usage  Balance node and network load  Minimize global network usage 2. Individual query performance  Minimize data delay  Maximize stream throughput 6

Network Usage Datarate = 50 MB/s Latency = 100 ms Datarate = 50 MB/s Latency = 30 ms A Datarate = 50 MB/s Latency = 40 ms B Datarate = 10 MB/s Latency = 200 ms Datarate = 10 MB/s Datarate = 50 MB/s Latency = 80 ms Latency = 50 ms Datarate = 75 MB/s In-flight Traffic: In-flight Traffic: Datarate = 75 MB/s Latency = 20 ms ∑ Datarate * Latency ∑ Datarate * Latency Latency = 100 ms = 5.8 MB = 17 MB 7

Network-Aware Operator Placement Perform operator placement in a decentralized fashion • Need information about data rate and latency But measuring network metrics is expensive • All pairs latency measurements are O(n 2 ) • Network latencies change over time • No global knowledge of measurements Idea : Approximate optimal query with a cost space [NetDB’05] 1. Build metric cost space to encode current network latencies 2. Find query with minimal network usage in cost space 3. Map query back to physical Internet nodes and instantiate 8

Cost Space Embed latency measurements into a metric space • Assign each SBON node a coordinate in a cost space • Euclidean distance ≈ network latency • Vivaldi algorithm [MIT]  Repeated measurements to refine local coordinate Advantages • Mathematical model for using geometric algorithms • Optimization decisions without global knowledge • Adaptive to change 9

Relaxation Placement Find a location for an operator that reduces network usage 10

Relaxation Placement Latency Datarate Use spring relaxation technique to find best location • Spring extension ≈ latency 11 • Spring constant ≈ data rate

Relaxation Placement Use spring relaxation technique to find best location • Springs “relax” to low energy state, minimizing network usage 12 • Dynamically adapts to changes in cost space

Relaxation Placement Uses nearest k-neighbor search for mapping of coordinates • Interesting problem in decentralized context 13  Geometric routing [HUJI] , DHT range queries [UCB] , …

Relaxation Placement Any SBON node can perform the placement for a new query • Local computation without global state  Inputs are coordinates of nodes and data rates in query • Supports placement of arbitrary complex queries  Model multiple queries as networks of spring Each node is then responsible for the operators it is hosting • Periodically re-execute Relaxation placement • Dynamically migrate operator to reflect new placement  Adapts to changes in latency and data rate 14

Simulation Setup Discrete event simulator to evaluate placement algorithms • GATech transit-stub topology with 1550 nodes  10 transit domains and 150 stub domains  Realistic Internet routing tables • 1000 queries with 5 random endpoints • Comparison of Relaxation placement 1KB/s to 4 other algorithms Optimal Exhaustive search Producer Common strategy Consumer Central data warehouse Random Worst case 15

Global Network Usage 100 90 80 Percentage of Queries Avg. Network Placement Usage 70 Algorithm Penalty 60 Optimal (NU) 0% 50 Relaxation 15% 40 Producer 43% 30 Consumer 60% 20 Random 81% 10 0 1000 1500 2000 2500 3000 3500 4000 4500 5000 5500 Network Usage (in KB) • Relaxation placement performs close to Optimal 16

Application Delay Penalty 100 90 Placement Avg. Delay Percentage of Queries 80 Algorithm Penalty 70 Optimal (NU) 13% Consumer 60 Relaxation 24% 50 Producer 75% 40 Consumer 0% Delay penalty: Random 76% 30 Longest path delay 20 IP delay 10 0 -50% 0% 50% 100% 150% 200% 250% 300% 350% Delay penalty • Consumer has smallest delay penalty • Relaxation has low delay penalty for an overlay network 17

Operator Migration on PlanetLab Relative Improvement in Network Usage Improved • 48 concurrent Network Usage queries on 130 nodes • ½ of the queries could migrate • Same initial place- ment for migrating and non-migrating queries Worse • Change in network Network Usage usage of migrating queries after 5 hours Query Pair Number • Migration decreased network usage for 75% of queries  17% less network usage and 11% lower application delay 18

Operator Reuse Share operators between overlapping sub-queries • Use cost space to bound search effort for reuse 19

Related Work Borealis [MIT, Brown, Brandeis] , Medusa [MIT] , Gates [Ohio] • Focus on high-availability and load management • Wide-area operator placement specified by user SAND [Brown] , PIER [UCB] • Operator placement at edge (prod/cons) or in-network • Exploit DHT routing paths for operator placement  Can lead to poor placement efficiency [IPTPS’05] IrisNet [Intel] • Hierarchical placement following DNS structure 20

Summary Large-scale stream applications need new systems support • SBON: Infrastructure for stream-processing applications • Provides network-aware stream query optimization Cost space approach for query optimization • Metric space for decentralised optimization decisions • Express query optimization as geometric problem Relaxation placement algorithm for operator placement • Scalable placement decisions reducing network usage • Continuous optimization as network conditions change Thank You. Any Questions? 21