Cost-Effective Resource Allocation for Deploying 1 Vinay Setty vsetty@mpi-inf.mpg.de Pub/Sub on Cloud Published in ICDCS 2014 Roman Vitenberg Maarten van Steen Gunnar Kreitz Guido Urdaneta
Cost-Effective Resource Allocation for Deploying 1 Vinay Setty vsetty@mpi-inf.mpg.de Pub/Sub on Cloud Published in ICDCS 2014 Roman Vitenberg Maarten van Steen Gunnar Kreitz Guido Urdaneta
Publish/Subscribe (Pub/Sub) 2
Publish/Subscribe (Pub/Sub) • Pub/sub is a communication paradigm 2
Publish/Subscribe (Pub/Sub) Publishers • Pub/sub is a communication paradigm Subscribers 2
Publish/Subscribe (Pub/Sub) Publishers • Pub/sub is a communication paradigm Middleware Subscribers 2
Publish/Subscribe (Pub/Sub) Publishers • Pub/sub is a communication paradigm Middleware Subscriptions Subscribers 2
Publish/Subscribe (Pub/Sub) Publishers • Pub/sub is a Publications communication paradigm Middleware Subscriptions Subscribers 2
Publish/Subscribe (Pub/Sub) Publishers • Pub/sub is a Publications communication paradigm • Pub/sub in the industry: • TIBCO rendezvous Middleware • GooPS: Google’s pub/ sub • Spotify’s pub/sub for Subscriptions social interaction [DEBS 2013] Subscribers 2
Application: Spotify Social Interaction Source: Wikimedia Commons ( CC BY-SA 3.0) 3
Application: Spotify Social Interaction • A music streaming service available in over 60 countries • Music events from users and artists result in Billions of notifications every day • Millions of users to be notified about these events at any time • Pub/sub for delivering music-related notifications [Setty et.al. DEBS'13] Source: Wikimedia Commons ( CC BY-SA 3.0) 3
Spotify Pub/Sub 4
Spotify Pub/Sub Real-time feed from friends and artists (fixed number of events) 4
Social Interaction in Spotify Spotify user 5
Social Interaction in Spotify Spotify Facebook Friend Friend Spotify user 5
Social Interaction in Spotify Spotify Facebook Friend Friend Spotify follow user 5
Social Interaction in Spotify Spotify Facebook Friend Friend Spotify follow user listened/starred track Playlist created/ updated 5
Social Interaction in Spotify Artists Spotify Facebook Friend Friend Spotify follow user listened/starred track Playlist created/ updated ✔ 5
Social Interaction in Spotify Artists Spotify Facebook Friend Friend Spotify follow user listened/starred follow track Playlist created/ updated ✔ 5
Social Interaction in Spotify Artists Spotify Facebook Friend Friend Spotify follow user listened/starred follow Album released track Playlist created/ updated ✔ 5
Social Interaction in Spotify Artists Spotify Facebook Friend Friend Spotify follow user listened/starred follow Album released track Playlist created/ l i s t e n e d t o updated t p r l a a c y k l i , s t ✔ a c t i v i t y 5
Social Interaction in Spotify Artists Spotify Facebook Friend Friend Spotify follow user listened/starred follow Album released track Playlist created/ l i s t e n e d t o updated t p r l a a c y k l i , s t ✔ a c t i v i t y Music Playlist 5
Social Interaction in Spotify Artists Spotify Facebook Friend Friend Spotify follow user listened/starred follow Album released track Playlist created/ l i s t e n e d t o updated t p r l a a c y k l i , s t ✔ a c t i v i t y follow Music Playlist 5
Social Interaction in Spotify Artists Spotify Facebook Friend Friend Spotify follow user listened/starred follow Album released track Playlist created/ l i s t e n e d t o updated t p r l a a c y k l i , s t ✔ a c t i v i t y Playlist updated follow Music Playlist 5
Social Interaction in Spotify Artists Spotify Facebook Friend Friend Spotify follow user listened/starred follow Topic Album released track Topic Playlist created/ Subscriber l i s t e n e d t o updated t p r l a a c y k l i , s t ✔ a c t i v i t y Playlist updated follow Music Playlist Topic 5
Social Interaction in Spotify Artists Spotify Facebook Friend Friend Spotify follow user listened/starred follow Topic Album released track Topic Playlist created/ Subscriber l i s t e n e d t o Pub/Sub for Social Interaction! updated t p r l a a c y k l i , s t ✔ a c t i v i t y Playlist updated follow Music Playlist Topic 5
Motivation Subscribers Publishers s n o i t a c fi i t o N Publications Datacenter/ Cloud 6
Motivation Subscribers Publishers s n o i t a c fi i t o N Publications Datacenter/ Cloud 6
Motivation Subscribers Publishers s n o i t a c fi i t o N Publications Datacenter/ Cloud What is the cost-effective way to allocate and distribute workload on multiple servers in a datacenter? 6
Motivation Subscribers Publishers s n o i t a c fi i t o N Publications Datacenter/ Cloud What is the cost-effective way to allocate and distribute workload on multiple servers in a datacenter? What is the minimum amount of resources needed to deploy pub/sub ? 6
Motivation Subscribers Publishers s n o i t a c fi i t o N Publications Datacenter/ Cloud What is the cost-effective What is the monetary cost way to allocate and distribute of deploying pub/sub on workload on multiple cloud ? servers in a datacenter? What is the minimum amount of resources needed to deploy pub/sub ? 6
Contributions 7
Contributions • Novel problem to minimize cost (e.g. monetary cost) of allocating pub/sub workload to servers in datacenters or cloud • Formalization and hardness analysis • Parametrized by generic pricing model from cloud providers 7
Contributions • Novel problem to minimize cost (e.g. monetary cost) of allocating pub/sub workload to servers in datacenters or cloud • Formalization and hardness analysis • Parametrized by generic pricing model from cloud providers • Efficient solutions for the formulated problem • Two stage heuristic-based solution • Several optimizations introduced 7
Contributions • Novel problem to minimize cost (e.g. monetary cost) of allocating pub/sub workload to servers in datacenters or cloud • Formalization and hardness analysis • Parametrized by generic pricing model from cloud providers • Efficient solutions for the formulated problem • Two stage heuristic-based solution • Several optimizations introduced • Extensive e valuation with real traces from Spotify and Twitter 7
How to Minimize Cost? 8
How to Minimize Cost? Social notifications have mostly human recipients [Gomez-Rodriguez, • ICWSM 2014] Introduce Satisfaction Delivery Rate beyond a threshold - not beneficial! • Threshold (rate limit ) Delivery Rate below a threshold - users unhappy! • Select and deliver a cost-effective subset of the workload so as to • meet satisfaction metrics while minimizing the cost [INFOCOM'14] 8
How to Minimize Cost? Social notifications have mostly human recipients [Gomez-Rodriguez, • ICWSM 2014] Introduce Satisfaction Delivery Rate beyond a threshold - not beneficial! • Threshold (rate limit ) Delivery Rate below a threshold - users unhappy! • Select and deliver a cost-effective subset of the workload so as to • meet satisfaction metrics while minimizing the cost [INFOCOM'14] Cost-effective allocation of resources • Given limited capacity of servers topic-subscriber pairs with shared • interests Goal is to minimize replication of events to different servers • Minimize number of servers deployed and bandwidth consumed • 8
Discussion 9
Discussion • In this work resource allocation was done based on events from social interactions 9
Discussion • In this work resource allocation was done based on events from social interactions • Can business processes help resource provisioning for social interaction applications in cloud? • Is targeted advertising a good application for integration of BPM and social events? 9
Recommend
More recommend
