PROST: A Programmable Structured Peer-to-Peer overlay Network - PowerPoint PPT Presentation

PROST: A Programmable Structured Peer-to-Peer overlay Network Marius Portmann, Sé ébastien Ardon bastien Ardon Marius Portmann, S Patrick Sé énac nac Patrick S University of Queensland, Brisbane, Australia ENSICA Toulouse, France

Context: overlay networks OVERLAY END-HOSTS NETWORK Prost: a Programmable structured P2P system

Structured peer-to-peer systems key space 0 128 2 chunk chunk Each node assigned and responsible for a chunk of key Each node assigned and responsible for a chunk of key space space Mapping of resources to key space done using a hash Mapping of resources to key space done using a hash function (e.g. SHA-1) function (e.g. SHA-1) – Most famous: Distributed Hash Tables Most famous: Distributed Hash Tables – Prost: a Programmable structured P2P system

Example: DHT with CHORD Circular Circular ID ID space space ( (wraps wraps) ) Node id IP@ … 7 Each node stores (key,values) for all keys Each node stores (key,values) for all keys 14 dest = 78 27 smaller than its node id but greater than smaller than its node id but greater than 7 50 4 previous node id. previous node id. 14 When node join/leave key,values are When node join/leave key,values are migrated to neighbor nodes migrated to neighbor nodes hash function is SHA1: nice uniform random hash function is SHA1: nice uniform random keyspace 27 output (load balancing) output (load balancing) Routing is used Routing is used to to find the node responsible find the node responsible for a particular key for a particular key 80 A node node uses « uses « shortcuts shortcuts » » along the along the ring: ring: A fingers fingers Fingers are Fingers are arranged exponentially starting arranged exponentially starting 60 from the node ID. ID. from the node Each node need to to find the closest find the closest Each node need 50 predecessor predecessor to to the the destination destination key key O(log N) messages exchanged exchanged for for routing routing O(log N) messages from any source to source to any any destination destination from any Each node export simple hash tables Each node export simple hash tables manipulations functions (get/put) manipulations functions (get/put) Prost: a Programmable structured P2P system

Structured P2P applications Cooperative File System (CFS) Cooperative File System (CFS) – File storage, uses CHORD at the block level – File storage, uses CHORD at the block level PAST PAST – large-scale, Internet-based, global storage utility that provides scalability, high – large-scale, Internet-based, global storage utility that provides scalability, high availability, persistence and security availability, persistence and security I3 I3 – Communication primitive (indirection) based on CHORD – Communication primitive (indirection) based on CHORD SCRIBE SCRIBE – scalable application-level multicast infrastructure, event notification infrastructure – scalable application-level multicast infrastructure, event notification infrastructure SplitStream SplitStream – High-bandwidth content distribution system based on application-level multicast – High-bandwidth content distribution system based on application-level multicast BAYEUX BAYEUX – application-level multicast built on top of Tapestry – application-level multicast built on top of Tapestry OCEANSTORE OCEANSTORE – – persistent and resilient data storage persistent and resilient data storage Prost: a Programmable structured P2P system

PROST idea Proliferation of new “ “DHT DHT” ”-based applications, each with -based applications, each with Proliferation of new their own topology their own topology – Re-inventing the wheel Re-inventing the wheel – One network per application! One network per application! – Overhead: topology maintenance Overhead: topology maintenance – Idea: share the topology, routing and topology Idea: share the topology, routing and topology maintenance between applications maintenance between applications – Common DHT API Common DHT API – Not enough: most applications need more than DHT Not enough: most applications need more than DHT Dynamically deploy application code on end-nodes Dynamically deploy application code on end-nodes (peers) (peers) – Take the programmable network paradigm to the peer-to-peer Take the programmable network paradigm to the peer-to-peer – layer layer – Allow to deploy new applications on a large scale, by leveraging – Allow to deploy new applications on a large scale, by leveraging an existing peer base an existing peer base Prost: a Programmable structured P2P system

A Layered model of structured p2p Dabek, F. et al, “Towards a common API for structured P2P overlays”, IPTPS’03 Prost: a Programmable structured P2P system

PROST Started as an enhanced DHT Started as an enhanced DHT – Adding functionalities to DHT Adding functionalities to DHT – – Problem to maintain consistency and versioning Problem to maintain consistency and versioning – Why not use DOLR? Why not use DOLR? – Objects residing on peers to provide application Objects residing on peers to provide application – functionalities functionalities must be previously deployed must be previously deployed Solution: dynamically deploy objects into peers, Solution: dynamically deploy objects into peers, on-demand on-demand Prost: a Programmable structured P2P system

Node architecture User User Interface Interface Component Component Peerlet Peerlet Peerlet Peerlet Manager Programmable Peer Layer Key-based Routing Layer (KBR ) Prost: a Programmable structured P2P system

Two types of applications Application Application (Peerlet) (Peerlet) KBR KBR KBR . . . (a) End-node applications: file sharing, storage, etc Application Application Application (Peerlet) (Peerlet) (Peerlet) KBR KBR KBR . . . (b) Per-hop applications: multicast communication, etc Prost: a Programmable structured P2P system

PROST messages peerlet dst key app id app type application data code locator DST KEY: destination key (ID) for this message DST KEY: destination key (ID) for this message APP ID: unique application-specific ID APP ID: unique application-specific ID APP TYPE: 1: end-node application, 0: per-hop APP TYPE: 1: end-node application, 0: per-hop application application – – Analogy to Router Alert IP header field Analogy to Router Alert IP header field Peerlet Code Locator: where to download the code if Peerlet Code Locator: where to download the code if needed needed Application Data: application-specific data, e.g. get/put Application Data: application-specific data, e.g. get/put operation and parameters operation and parameters Prost: a Programmable structured P2P system

PROST API Based on Dabek Dabek et al. et al. “ “Towards a common API Towards a common API Based on for structured P2P overlays” ” for structured P2P overlays send a message ): send a message – route(k, route(k,msg msg, ,nextHopHint nextHopHint): – to destination key k to destination key k upcall to peerlet at ): upcall to peerlet at – forward(k, forward(k,msg msg, ,nextHop nextHop): – intermediate nodes intermediate nodes – deliver(k, deliver(k,msg msg) ) – – Plus access to local routing information: Plus access to local routing information: – local_lookup( (… …), ), neighbour_set neighbour_set( (… …), ), local_lookup replicaSet( (… …), update( ), update(… …), range( ), range(… …) ) replicaSet Prost: a Programmable structured P2P system

Typical operation dst key = 78 App ID = ‘P2PTV v1.9’ App TYPE = ‘1’ Peerlet code locator = ‘https://secure.p2ptv.com/cgi?code-current P2PTV App DATA: addReceiver(4,…) route (80,60,appData) peerlet 7 4 14 deliver (78,appData) forward (78,appData,60) 27 download and PROST NETWORK install peerlets 80 no 60 yes yes appID appType=1? 50 installed? Prost: a Programmable structured P2P system

Node security policies Security policies are configured in each Security policies are configured in each node node – allow/deny peerlets to be installed, using allow/deny peerlets to be installed, using – peerlet provider identity peerlet provider identity application ID application ID local policy (deny/allow all) local policy (deny/allow all) Allow graceful degradation of the service Allow graceful degradation of the service – if peerlet denied installation, try first node in if peerlet denied installation, try first node in – replicaSet (recursively) (recursively) replicaSet Prost: a Programmable structured P2P system

Challenges Similar to programmable networks: Similar to programmable networks: – resource management resource management – admission control admission control – security security – trust model, code authentication trust model, code authentication sandboxing sandboxing use to Crypto-based IDs use to Crypto-based IDs New to p2p-based “ “active active” ” networking networking New to p2p-based – dynamicity of execution environment: nodes join and – dynamicity of execution environment: nodes join and leave leave process migration? process migration? Prost: a Programmable structured P2P system