Heterogeneous Gossip Davide Frey Rachid Guerraoui Anne-Marie Kermarrec Boris Koldehofe Maxime Monod Martin Mogensen Vivien Quéma
Outline • Context – Live Streaming – Gossip – Limitations • Heterogeneous Gossip – Protocol – Evaluation – Conclusion
Context • Large-scale (no IP-multicast) • Churn/Failures • Constrained, asymmetric and heterogeneous bandwidth Target application: Live streaming
Live Streaming Sent at t s Received at t r t 0 t 1 t 2 t 3 ? Play the stream at t r +δ Maximize stream quality Minimize δ ( buffering delay ) Play the stream at t 0 +ε Minimize ε ( stream lag )
Existing approaches Structured overlay Unstructured overlay Static overlay / Reactive repair Dynamic overlay / Proactive repair DHT-based systems Gossip Trees Mesh-based systems Multiple trees Trees over mesh
Existing approaches Structured overlay Unstructured overlay Static overlay / Reactive repair Dynamic overlay / Proactive repair SplitStream (2003) BAR Gossip/FlightPath (2006/2008) Chunkyspread (2006) Coolstreaming (2005) Gossip Chainsaw (2005) PULSE (2007) PRIME (2007) Coolstreaming (2007) GridMedia (2007) Heterogeneity awareness?
Gossip in the real world George meets Bob:
Gossip in computer science fanout partners Infect-and-die model p 0 p 1 p 2 p 3 p 4 … p f gossip period
Gossip - Theory 1. fanout = ln( n ) + c P[connected graph] goes to exp(-exp(- c )) 1 c=3 → 95% c=2 → 87% 0.8 c=1 → 69% 0.6 0.4 c=0 → 37% 0.2 c=-1 → 7% 0 ln(n)-10 ln(n)-5 ln(n) ln(n)+5 ln(n)+10 2. Holds as long as the fanout is ln( n ) + c on average
Gossip – Practice (600kbps) Percentage of nodes receiving at least 99% of the stream 100 90 80 Percentage of nodes (CDF) 70 60 50 PlanetLab nodes have: 40 30 Large bandwidths • 20 Small delays • 10 0 0 5 10 15 20 25 30 35 40 45 50 55 60 Stream lag (s)
Gossip is load-balancing… S p 3 p 2 Proposals arrive randomly – Nodes pull from the first proposal p 1 • q Highly-dynamic – S q S q Node q will serve f nodes whp Node q will serve f nodes wlp
… but the world is heterogeneous! Load-balancing Capability Percentage of nodes receiving at 3 classes (691kbps avg): least 99% of the stream 100 90 512kbps 80 Percentage of nodes (CDF) 85% 70 60 50 40 3Mbps 1Mbps 30 20 5% 10% 10 0 0 5 10 15 20 25 30 35 40 45 50 55 60
How to cope with heterogeneity? • Goal: contribute according to capability • Advertize more = sell more: vs – Propose more = serve more • Increase fanout… … and decrease it too! • Challenges: – Preserve reliability of gossip average fanout (f avg ) ≥ initial fanout = ln( n ) + c – Cope with dynamic capabilities
Heterogeneous Gossip - HEAP Capability • q and r with bandwidths b q > b r – q should upload b q / b r times as much as r • Who should increase/decrease its contribution? … and by how much? • How to ensure reliability? – How to keep f avg constant?
HEAP b avg Capability • Total/average contribution is equal in both homogeneous and heterogeneous settings f q = f init ∙ b q / b avg …ensuring the average fanout is constant and equal to f init = ln( n ) + c
HEAP • Get b avg with (gossip) aggregation – Advertize own and freshest received capabilities – Aggregation follows change in the capabilities • Get n with (gossip) size estimation – Estimation follows change in the system • Join/leave • Crashes • …
Stream Lag reduction Percentage of nodes receiving at least 99% of the stream 100 90 80 Percentage of nodes (CDF) No cap 70 60 HEAP – 691kbps 50 Standard gossip – 691kbps 40 30 20 10 0 0 5 10 15 20 25 30 35 40 45 50 55 60 Stream lag (s)
Experimental Setup • 270 PlanetLab nodes • Network capabilities – Bandwidth cap by throttling – Communication with UDP • Stream rate of 600kbps – Windows of 110 events, including 9 FEC events • Gossip – period of 200 ms – f avg = 7 ( ln (270) = 5.60)
Evaluation Metrics • Stream Lag – Time difference between creation at the source and delivery to the player • Stream Quality – A window is considered jittered if < 101 events – Stream with maximum of 1% jitter means at least 99% of the windows are complete • Incomplete does not mean “blank”!
Quality improvement • Stream lag of 10s Jitter-free percentage of the stream 100 90 512kbps 80 1Mbps 70 3Mbps 60 50 40 30 20 10 0 Standard Gossip HEAP
Stream Lag • For those who can have a jitter-free stream Average stream lag to obtain a jitter-free stream 45 512kbps 40 1Mbps 35 Stream lag (s) 3Mbps 30 25 20 15 10 5 0 Standard Gossip HEAP
Proportional contribution Average bandwidth usage by bandwidth class 3072 512kbps 1Mbps 2560 3Mbps 2048 87.56% 1536 99.89% 94.38% 1024 48.44% 91.56% 90.58% 512 0 Standard Gossip HEAP
20% nodes crashing Failure of 20% of the nodes at time t=60s 100 Percentage of nodes receiving each window 80 60 40 20 HEAP - 12s lag Standard Gossip - 20s lag Standard Gossip - 30s lag 0 0 30 60 90 120 150 Stream time (s)
Conclusion • Limitations – UDP usage • TCP-unfriendliness • Incoming traffic – Probability of acceptance also depends on latency • Future work – Compare with mesh systems – Freeriders – Biasing partner selection
Recommend
More recommend
