AmazingStore: Available, Low-cost Online Storage Service Using - PowerPoint PPT Presentation

AmazingStore: Available, Low-cost Online Storage Service Using Cloudlets Ben Y. Zhao Zhi Yang, U. C. Santa Barbara Yuanjian Xing, Song Ding, Feng Xiao Yafei Dai Peking University

Motivation  Online storage services are getting increasingly popular  Amazon’s S3, EMC’s Mozy …  Rely on data centers.  Challenges  Threatened by the single point of failure.  Amazon suffers outages (3 times); Gmail is down (4 times) …  Social networks make downtime harder to hide.  Incur high hardware, network and cooling costs.  P2P storage  Use idle resource of users to avoid costs  Provide low availability because of churn.

Motivation  Best of both worlds  Stability of data center  Low cost of P2P Data Center P2P Storage Layer

AmazingStore Design  Combine data center and P2P storage system Location of replicas Master DHT Master Location of replicas

AmazingStore Design  During data center outage  Degrade to pure P2P storage  Peers closest to servers are assigned as new master nodes. Location of replicas Master DHT Master Location of replicas

AmazingStore Design  Determine replication degree at the peer layer  Threshold c derived from hybrid availability model Data center Target availability Availability     log(1 ) log(1 ) A  S c   log(1 ) P Peer availability

AmazingStore Design  Maintain replication level at the peer side  Probabilistic failure detection Pr(permanent|offline 5 hours)  Group-level estimator of replica number = 0.1 Transient Peer A Possible situations Pr(A and B are transient) = 0.36 Peer B Pr(permanent|offline 25 hours) =0.6 Pr(one is transient, the other is permanent) = 0.58 Permanent Pr(A and B are permanent) = 0.06 Peer C

AmazingStore: Sharing & Storage  Upload files important to you File Storage Sharing Files with guarantee locally with DHT SLA: No Availability guarantee guarantee Upload files

Preliminary Measurements  composed of users and data center containing PKU servers.  As of early April. 2010  Registered users >11,820  Daily peak of online users > 1000  Data objects > 52,055.  Provides a target of two nines availability  The data center availability is only 0.932  maintain at least 6 replicas at peer side.

Availability Improvement  Overall availability jumps from 93.22% to 99.13%  Availability gained at peer side is 83.8% 1 Fraction of available objects P2P layer 0.8 compensation 0.6 0.4 Server-side Availability Peer-side Availability 0.2 System Availability 0 05/25 05/26 05/27 05/28 Date Power failure

Bandwidth offloading  90.38% requests were handled by the peer layer.  Average download bandwidth is 2.1MB/sec 1 Peer-side 0.8 offloading Fraction of requests Satisfied on the peer side 0.6 Satisfed on the server side 0.4 0.2 0 06/09 06/11 06/13 06/15 06/17 06/19 06/21 Date

Discussion  Edge-cached system cannot work alone. Only cached replicas 40 30 Replica count 20 10 6 0 0 1000 2000 3000 4000 5000 6000 Rank Fall below the target

Discussion  AmazingStore works well with data recovery Cached replicas + Repaired replicas 40 30 Guarantee Replica count enough replicas 20 10 6 0 0 1000 2000 3000 4000 5000 6000 Rank

Conclusion  We advocate that data center and peers can complement well.  We describe a deployed prototype called AmazingStore.

Questions ? Thank you!