Data Storage Solutions for Decentralized Online Social Networks - PowerPoint PPT Presentation

Data Storage Solutions for Decentralized Online Social Networks — Anwitaman Datta S* Aspects of Networked & Distributed Systems (SANDS) � School of Computer Engineering 
 NTU Singapore iSocial Summer School, KTH Stockholm

Research @ SANDS Applica'ons) recommenda7on&and& decentralized&online&social& decision&support&systems& networking&and&collabora7on& & & distributed&key:value&stores& (Distributed)))Systems) privacy&aware/preserved&data& data:center& P2P/F2F& aggrega7on,&storage,&sharing&& design& storage& &&analy7cs/data:mining& & systems& networked&distributed&storage&& data/computa7on&&at&& &&data&management&systems& 3 rd &party/outsourced& Founda'onal) social& trust& secure/privacy& codes&for& network& models& preserved&computa7on& storage& analysis& & primi7ves& &

DOSNish research at SANDS

DOSNish research at SANDS Selective information dissemination using social links GoDisco

DOSNish research at SANDS Selective information dissemination using social links GoDisco Security issues Access control, Private Information Retrieval, …

DOSNish research at SANDS Selective information dissemination using social links GoDisco Security issues Access control, Private Information Retrieval, … DOSN architectures PeerSoN, SuperNova, PriSM, …

DOSNish research at SANDS Selective information dissemination using social links GoDisco Security issues Access control, Private Information Retrieval, … DOSN architectures PeerSoN, SuperNova, PriSM, … P2P storage

DOSNish research at SANDS Selective information dissemination using social links GoDisco Security issues Access control, Private Information Retrieval, … DOSN architectures PeerSoN, SuperNova, PriSM, … P2P storage h"p:// sands .sce. ntu .edu.sg/0

P2P Storage Not the same as a file-sharing system Peer-to-Peer (P2P) storage systems leverage the combined storage capacity of a network of storage devices (peers) contributed typically by autonomous end-users as a common pool of storage space to store content reliably.

P2P Storage

P2P Storage Design space

P2P Storage Design space Reliability: Availability & Durability (focus of this talk)

P2P Storage Design space Reliability: Availability & Durability (focus of this talk) Security & Privacy: Access control, integrity, free- riding, anonymity, privacy, …

P2P Storage Design space Reliability: Availability & Durability (focus of this talk) Security & Privacy: Access control, integrity, free- riding, anonymity, privacy, … Sophisticated functionalities: Concurrency, Version Control, …

Realizing Reliability Garbage collection Maintenance strategies Lazy: Randomized P2P#storage# Diversity of Reactive design#space# online fragments Lazy: Deterministic (Threshold based) Eager: Repair all Duplicates of same fragment Proactive New codes, Erasure codes Replication e.g. self-repairing codes Redundancy type Key based (e.g., DHTs) Selective (e.g., at friends or trusted nodes , history or proximity based, etc.) Random Placement

Redundancy Type

Redundancy Type Replication

Redundancy Type Replication Erasure codes B 1 Retrieve any O 1 O 1 k’ ( ≥ k) blocks B 2 … O 2 Reconstruct Data O 2 Data = Object … … B l Decoding Encoding … O k O k Lost blocks Original k blocks B n k blocks n encoded blocks (stored in storage devices in a network)

Redundancy placement

Redundancy placement A rather complicated problem All peers are fully cooperative and altruistic, but autonomous System capacity and resource allocation … • Heterogeneity, … Coverage: history/prediction/…

Redundancy placement A rather complicated problem All peers are fully cooperative and altruistic, but autonomous System capacity and resource allocation … • Heterogeneity, … Coverage: history/prediction/… Selfish/Byzantine peers: Incentives, trust, enforcement, …

Redundancy placement A rather complicated problem All peers are fully cooperative and altruistic, but autonomous System capacity and resource allocation … • Heterogeneity, … Coverage: history/prediction/… Selfish/Byzantine peers: Incentives, trust, enforcement, … Security & privacy implications of data placement …

Classical P2P storage systems Successor$list$ replicas) DHT$ID$space$

Classical P2P storage systems Distributed Hash Table (DHT) determines storage placement, e.g., CFS/ OpenDHT Successor$list$ replicas) DHT$ID$space$

Classical P2P storage systems Distributed Hash Table (DHT) determines storage placement, e.g., CFS/ OpenDHT Successor$list$ Pros: Simple design, ease of locating data replicas) DHT$ID$space$

Classical P2P storage systems Distributed Hash Table (DHT) determines storage placement, e.g., CFS/ OpenDHT Successor$list$ Pros: Simple design, ease of locating data Cons: mixes indexing with storage replicas) DHT$ID$space$

Classical P2P storage systems Distributed Hash Table (DHT) determines storage placement, e.g., CFS/ OpenDHT Successor$list$ Pros: Simple design, ease of locating data Cons: mixes indexing with storage high correlation of failures replicas) DHT$ID$space$

Classical P2P storage systems Distributed Hash Table (DHT) determines storage placement, e.g., CFS/ OpenDHT Successor$list$ Pros: Simple design, ease of locating data Cons: mixes indexing with storage high correlation of failures replicas) cannot leverage other 
 characteristics DHT$ID$space$ • e.g., locality, history, etc.

Classical P2P storage systems Distributed Hash Table (DHT) determines storage placement, e.g., CFS/ OpenDHT Successor$list$ Pros: Simple design, ease of locating data Cons: mixes indexing with storage high correlation of failures replicas) cannot leverage other 
 characteristics DHT$ID$space$ • e.g., locality, history, etc. may lead to poor performance • access latency, repair cost, …

Classical P2P storage systems $ t s i l $ r o s s e c c u S pointers)to)) replicas) DHT$ID$space$

Classical P2P storage systems Distributed Hash Table (DHT) as a directory, e.g., TotalRecall $ t s i l $ r o s s e c c u S pointers)to)) replicas) DHT$ID$space$

Classical P2P storage systems Distributed Hash Table (DHT) as a directory, e.g., TotalRecall Pros: Flexible placement policy $ t s i l $ r o s s e c c u S pointers)to)) replicas) DHT$ID$space$

Classical P2P storage systems Distributed Hash Table (DHT) as a directory, e.g., TotalRecall Pros: Flexible placement policy Cons of TotalRecall, which placed at random: $ t s i l $ r o s s e c c u S ??? pointers)to)) replicas) DHT$ID$space$

Cloud assisted storage system Source: ¡Google ¡tech ¡talk ¡on ¡Wuala: ¡http://www.youtube.com/watch?v=3xKZ4KGkQY8 ¡

Cloud assisted storage system Hybrid architecture (used previously in Wuala) Source: ¡Google ¡tech ¡talk ¡on ¡Wuala: ¡http://www.youtube.com/watch?v=3xKZ4KGkQY8 ¡

Cloud assisted storage system Hybrid architecture (used previously in Wuala) Superpeers DHT Source: ¡Google ¡tech ¡talk ¡on ¡Wuala: ¡http://www.youtube.com/watch?v=3xKZ4KGkQY8 ¡

Cloud assisted storage system Hybrid architecture (used previously in Wuala) Superpeers DHT Users Source: ¡Google ¡tech ¡talk ¡on ¡Wuala: ¡http://www.youtube.com/watch?v=3xKZ4KGkQY8 ¡

Cloud assisted storage system Hybrid architecture (used previously in Wuala) Storage ¡peers Superpeers DHT Users Source: ¡Google ¡tech ¡talk ¡on ¡Wuala: ¡http://www.youtube.com/watch?v=3xKZ4KGkQY8 ¡

Cloud assisted storage system Hybrid architecture (used previously in Wuala) Storage ¡peers Superpeers DHT G E T Users Source: ¡Google ¡tech ¡talk ¡on ¡Wuala: ¡http://www.youtube.com/watch?v=3xKZ4KGkQY8 ¡

Cloud assisted storage system Hybrid architecture (used previously in Wuala) Storage ¡peers Superpeers DHT Routing G E T Users Source: ¡Google ¡tech ¡talk ¡on ¡Wuala: ¡http://www.youtube.com/watch?v=3xKZ4KGkQY8 ¡

Cloud assisted storage system Hybrid architecture (used previously in Wuala) Storage ¡peers Superpeers DHT Routing G E T Users Source: ¡Google ¡tech ¡talk ¡on ¡Wuala: ¡http://www.youtube.com/watch?v=3xKZ4KGkQY8 ¡

Cloud assisted storage system Hybrid architecture (used previously in Wuala) Storage ¡peers Superpeers DHT Routing G E T Users Source: ¡Google ¡tech ¡talk ¡on ¡Wuala: ¡http://www.youtube.com/watch?v=3xKZ4KGkQY8 ¡

Cloud assisted storage system Hybrid architecture (used previously in Wuala) Storage ¡peers Superpeers DHT Routing G E T Users Source: ¡Google ¡tech ¡talk ¡on ¡Wuala: ¡http://www.youtube.com/watch?v=3xKZ4KGkQY8 ¡