Contents • Introduction • Basic Model High Availability, Scalable Storage, • Availability and Redundancy Dynamic Peer Networks: Pick Two • Discussion • High Availability, Scalable Storage, Nov. 24, 2003 Dynamic Peer Networks: Pick Three Byung-Gon Chun 1 2 Introduction Basic Model • Peer-to-peer lookup: robust, scalable with • Assumptions dynamic membership ⇒ – Simple redundancy maintenance mechanism (enter and exit) Robust and scalable storage with dynamic – Static data placement strategy (f: RB-> N) membership ? – Identical per-node space and bandwidth contributions • Pick two – Constant rate of entering and exiting. – Lookup is not bottleneck. – Independence of exit events – (upstream) bandwidth limitation – Constant steady-state number of nodes and total data – Disk space grows faster than access bandwidth size – Maintenance bandwidth • Average case analysis 3 4
Basic Model Understanding the Scaling • N: number of hosts (k = 20) • D: data • S: data + redundancy (S = kD) • α : entering rate • λ : exiting rate ( α = λ ) • T: lifetime (T=N/ λ ) • B: bandwidth - Short membership : enormous nodes to scale - How fast storage of systems can grow? 5 6 Availability & Redundancy Model • Membership timeout: distinguish true departures from temporary downtime, delay its response to failures • Counting offline hosts as members – Lifetime is longer – Hosts serve as a fraction of time ( a : availability) – More redundancy is needed – Effective bandwidth is reduced • Redundancy: replication vs. erasure coding 7 8
Availability & Redundancy Membership Timeout • 33000 hosts Gnutella network, 1TB data, six nine data availability • 30-fold savings by membership timeout • Additional 8-fold savings by erasure coding – 75Kbps maintenance bandwidth per node – 500MB of disk per host contributed • 5000 of 33000 hosts usually available – Aggregate bandwidth 500Mbps – 5 dedicated, reliable PCs with 250GB drives and 50Mbps connection up 99% of the time 9 10 Admission Control, Load- Replication vs. Coding Shifting • Do not admit highly volatile nodes, Shift responsibility to non-volatile hosts • 5% most available hosts - 40% of service years. – 30Kbps per node per unique-TB using coding – 1000-fold savings using delayed response, coding, and admission control • Still bounded by bandwidth – 100Kbps maintenance bandwidth, 3GB disk space – 10 universities with 1/3 OC3 • Two million cable modem users at 40% availability ~ 2000 universities with ½ OC3 11 12
Hardware Trends Incentive Issues • Stable membership is necessary. • How to incent? – Added value of service guarantees – Allow client bandwidth usage to be only proportional to contributed bandwidth -- Prioritizing traffic • Participation should be more stable to contribute meaningful fraction of disks 13 14 Discussion Discussion • High availability, scale, dynamic • Why worry about lookup guarantees if membership: high service bandwidth storage guarantees are inappropriate? ⇒ Current DHT research trajectory ??? • When anonymity or related security properties are the high, why not plan to • Static membership – small lookup-state include the defense from the beginning? optimization do more harm than good (another approach - one-hop lookup) • Dynamic membership – why leverage many flaky nodes to serve data a few reliable ones (another approach – distributed directory) 15 16
Availability Pick Three • Distributed directory (DD) – Uses a level of indirection – Controls the data placement – Exploits heterogeneity (availability, lifetime, and bandwidth) [Bhagwan, Savage, and Voelker 2003] Pick Three!!! 17 18 Discussion? 19
Recommend
More recommend
