Experiences with CoralCDN A Five‐Year Opera:onal View Michael J. Freedman Princeton University www.coralcdn.org
A Coopera:ve, Self‐Organizing CDN Client Resolver 2 5 1 CoralCDN CoralCDN CoralCDN CoralCDN HTTP Proxy DNS Server HTTP Proxy DNS Server Coral index node Coral index node 3 6 4 CoralCDN CoralCDN CoralCDN HTTP Proxy DNS Server CoralCDN HTTP Proxy Coral index node Coral index node Goal: To make desired content widely available regardless of publisher’s own resources, by organizing and u:lizing any coopera:ve resources
hPp://example.com/path hPp://example.com.nyud.net/path
Adopted by: Clients Servers Third‐par:es
Many of you have used CoralCDN
Many of you have used CoralCDN
Many of you have used CoralCDN
Many of you have used CoralCDN
Many of you have used CoralCDN
Many of you have used CoralCDN
From Clients 100 Requests per Day (Millions) To Upstream Proxy/Origin 10 1 0.1 Jan’05 Jan’06 Jan’07 Jan’08 Jan’09 Jan’10 Daily Request Volume 2M clients – 2 TB content – 20K origin domains From 300‐400 PlanetLab servers
CoralCDN CoralCDN CoralCDN CoralCDN HTTP Proxy DNS Server HTTP Proxy DNS Server Coral index node Coral index node CoralCDN CoralCDN CoralCDN HTTP Proxy DNS Server CoralCDN HTTP Proxy Coral index node Coral index node Based on peer‐to‐peer DHT 1. Weakened consistency + algorithms that prevent tree satura:on during lookup 2. Decentralized clustering for locality and hierarchical lookup 3. Coopera:ve HTTP / DNS that leverages locality
CoralCDN CoralCDN CoralCDN CoralCDN HTTP Proxy DNS Server HTTP Proxy DNS Server Coral index node Coral index node CoralCDN CoralCDN CoralCDN HTTP Proxy DNS Server CoralCDN HTTP Proxy Coral index node Coral index node Based on peer‐to‐peer DHT 1. Weakened consistency + algorithms that prevent tree satura:on during lookup 2. Decentralized clustering for locality and hierarchical lookup 3. Coopera:ve HTTP / DNS that leverages locality
CoralCDN CoralCDN CoralCDN CoralCDN HTTP Proxy DNS Server HTTP Proxy DNS Server Coral index node Coral index node CoralCDN CoralCDN CoralCDN HTTP Proxy DNS Server CoralCDN HTTP Proxy Coral index node Coral index node Interac:ons with the Virtualiza:on Layer External Environment Clients Origin Domains
1. Experiences – Naming – Fault Tolerance – Resource management 2. Revisit CoralCDN’s design
Naming Flexible, open API x Mismatch with domain‐based access control policies
CoralCDN’s Plaaorm‐as‐a‐Service API Rewrite rules in origin webservers RewriteEngine on RewriteCond %{HTTP_USER_AGENT} !^CoralWebPrx RewriteCond %{QUERY_STRING} !(^|&)coral-no-serve$ RewriteRule ^(.*)$ http://%{HTTP_HOST}.nyud.net% {REQUEST_URI} [R,L]
CoralCDN’s Plaaorm‐as‐a‐Service API Rewrite rules in origin webservers RewriteEngine on RewriteCond %{HTTP_USER_AGENT} !^CoralWebPrx RewriteCond %{QUERY_STRING} !(^|&)coral-no-serve$ RewriteCond %{HTTP_REFERER} slashdot\.org [NC] RewriteCond %{HTTP_REFERER} digg\.com [NC,OR] RewriteCond %{HTTP_REFERER} blogspot\.com [NC,OR] RewriteRule ^(.*)$ http://%{HTTP_HOST}.nyud.net% {REQUEST_URI} [R,L] Sites integrate with load/bandwidth monitoring Elas:c Provisioning
Naming Confla:on hPp://domain /path .service1 .service2 1. Loca:on to retrieve content x 2. Human‐readable name for administra:ve en:ty x 3. Security policies to govern objects’ interac:ons
Domain‐based Security Policies Web Page evil.com target.com Cookies Document Object Model
Domain‐based Security Policies Web Page evil.com .nyud.net target.com .nyud.net Cookies Document Object Model Defaults violate least privilege
Fault Tolerance: Failure Decoupling Internal failures: • DHT nodes • DNS servers, HTTP proxies • Management service x External failures: • Decouple IPs from hosts • Interac:ons with origin sites
happens! Origin Status CoralCDN ReacAon 1. Unresponsive • Cache nega:ve results 2. Returns error code • Serve stale content 3. Reply truncated • Use whole‐file overwrites
happens! Origin Status CoralCDN ReacAon 1. Unresponsive • Cache nega:ve results 2. Returns error code • Serve stale content 3. Reply truncated • Use whole‐file overwrites Maintain status quo unless improvements are possible
What is “failure”? Return values should have fail‐safe defaults
Resource Management Control over bandwidth consump:on x Control and visibility into environment’s resources
Some :meline… Mar 2004 CoralCDN released on PlanetLab
Some :meline… Mar 2004 Aug 2004 CoralCDN SlashdoPed released on PlanetLab
Some :meline… Mar 2004 Aug 2004 Dec 2004 CoralCDN SlashdoPed Asian released on Tsunami PlanetLab 1. PlanetLab traffic jumps 2. Site threatens to yank PL 3. PL admin kills slice 4. Slice restored next day 5. Ini:ates discussion of resource limits for slices
Demand >> Supply: Enter Fair‐Sharing Algorithms Avg MB per hour (d i ) ? Σ i d i ≤ S Domains with heaviest consump:on
Demand >> Supply: Enter Fair‐Sharing Algorithms Avg MB per hour (d i ) find max λ , s.t. Σ i min ( λ , d i ) ≤ S λ Domains with heaviest consump:on
Demand >> Supply: Enter Fair‐Sharing Algorithms find max λ , s.t. Σ i min ( λ , d i ) ≤ S λ Domains with heaviest consump:on
Admission Control under Fair‐Sharing ~10 kB imgs ~5 MB videos 3.3% rejected 89% rejected Requests per Domain Requests per Domain Requests per Domain 1e+06 1e+06 1e+06 All Responses All Responses All Responses Forbidden Responses 100000 100000 100000 10000 10000 10000 1000 1000 1000 100 100 100 10 10 10 1 1 1 1 1 1 10 10 10 100 100 100 1000 1000 1000 10000 10000 10000 Unique Domains Ordered by Decreasing Popularity Unique Domains Ordered by Decreasing Popularity Unique Domains Ordered by Decreasing Popularity Demand > 10 TB Supply ≤ 2 TB
Some :meline… Mar 2004 Aug 2004 Dec 2004 Mar 2006 CoralCDN SlashdoPed Asian PL deploys released on Tsunami bandwidth PlanetLab throPling 1. PlanetLab traffic jumps 2. Site threatens to yank PL 3. PL admin kills slice 4. Slice restored next day 5. Ini:ates discussion of resource limits for slices
Resource Management: Us vs. Them ApplicaAon Hammer PlaEorm Hammer • Track HTTP traffic • Track all network traffic • If site > fair share rate, • If total > 80% daily rate, reject via HTTP 403 BW shaping in kernel • If total > peak rate, close server socket
Resource Management: Us vs. Them ApplicaAon Hammer PlaEorm Hammer • Track HTTP traffic • Track all network traffic • If site > fair share rate, • If total > 80% daily rate, reject via HTTP 403 BW shaping in kernel • If total > peak rate, close server socket Result: HTTP traffic is 1/2 ‐ 2/3 of all traffic Lower layers should expose greater visibility and control over resources
1. Experiences – Naming – Fault Tolerance – Resource management 2. Revisit CoralCDN’s design
Recommend
More recommend
