Automatic Deployment of Peer-to-Peer Network Infrastructures PhD - PowerPoint PPT Presentation

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions Automatic Deployment of Peer-to-Peer Network Infrastructures PhD Scientific Report R˘ azvan Deaconescu Supervisor: prof. dr. ing. Nicolae T , ˘ apus , Computer Science and Engineering Department January 31, 2011, Bucharest R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions Outline 1 Context 2 Virtualized Infrastructure 3 Automated Framework 4 Data Acquisition 5 Setup and Results 6 Conclusion 7 Questions R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions Peer-to-Peer Systems Peer-to-Peer vs. client-server continuous evolution since 2000 (Napster) definition Does it treat variable connectivity and temporary addresses as a natural occurrence? Does it provide autonomy to all nodes, even those on the edge of the network? streaming technology making its way R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions BitTorrent flagship of BitTorrent most core traffic is still BiTorrent HTTP rapidly catching up ( video streaming ) multiple implementations rarest-piece-first, tit-for-tat, optimistic-unchoke R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions Measurements in Peer-to-Peer Systems testbeds (PlanetLab) simulations real world scenarios data collection and storage trackers R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions Virtualization hardware/server virtualization multiple instances of separate OSs running on same hardware hypervisor (VMM) consolidation, reliability, security full virtualization, paravirtualization, OS-level virtualization R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions Goals automated infrastructure for BitTorrent clients scalable and flexible (virtualization) complete control over swarm environment (private infrastructure) intensive client information (client logging) performance evaluation and comparison internal protocol analysis R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions Outline 1 Context 2 Virtualized Infrastructure 3 Automated Framework 4 Data Acquisition 5 Setup and Results 6 Conclusion 7 Questions R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions Virtualization Solutions constraints: Linux environment, automation, low overhead KVM: native virtualization, not easily automated built, some overhead Xen: paravirtualization (good performance), separation, overhead OS-level virtualization LXC: mainline kernel support, lack of documentation OpenVZ: good spread, good documentation R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions OpenVZ OS-level virtualization low overhead (extended chroot) – runs directly on top of operating system templates CLI-interaction ( vzctl ) – automation resource limitation (user beancounters) R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions Overall View R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions Tools Employed vzctl + templates shell scripting SSH tc (traffic control) iptables brctl sysstat R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions Virtualization Evaluation efficiency (scalability) – how many virtual machines/containers may be deployed on a virtual host and allow proper simulation of an environment; isolation – how well are virtual machines’ resources separated; reliability – how many software crashes happen for a given solution; this may be due to implementation or to resourse overuse/abuse. R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions Virtualization Evaluation (2) Eff = f ( HW , SW , VS ) (1) Eff = f ( RAM , HDD , CPU , NET , OS , PS , BT , VS , NVM ) (2) Eff = VMB (3) HNB Iso ( normalprocesses ) < Iso ( chroot ) < Iso ( OpenVZ , LXC ) < Iso ( Xen , KVM ) (4) Rel = f ( HW , SW , VS ) (5) Rel = f ( RAM , HDD , CPU , OS ( filesystem ) , PS , BT , VS , NVM ) (6) R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions Open Issues formal evaluation of virtualization solutions and BitTorrent processes LXC vs. OpenVZ OS-level virtualization efficiency/scalaibility traffic control in OpenVZ scalability evaluation R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions Outline 1 Context 2 Virtualized Infrastructure 3 Automated Framework 4 Data Acquisition 5 Setup and Results 6 Conclusion 7 Questions R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions Architecture XML Configuration Files OpenVZ Container bootstrap Server (SSH) Tribler command Server Hrktorrent a c k / e r r o r Transmission Commander Station R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions Goals automation complete control (internal swarms vs. external swarms) full client information R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions Client Instrumentation open-source clients update for status logging instrument for verbose logging wrappers for starting/stopping/collecting results R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions Client Management Commander station service running on each container (start, stop, status) specialized protocol XML-based node-description and swarm-description R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions Outline 1 Context 2 Virtualized Infrastructure 3 Automated Framework 4 Data Acquisition 5 Setup and Results 6 Conclusion 7 Questions R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures

Context Virtualized Infrastructure Automated Framework Data Acquisition Setup and Results Conclusion Questions Client Logging status messages periodic (usually each second) client state upload speed, download speed, eta, download percentage lightweight may be monitored verbose log messages per-protocol/client behavior internal events (chokes, unchokes, have, piece transmission) heavyweight cannot be monitored to be collected, parsed and disseminated R˘ azvan Deaconescu CSE Dep., ACCF, UPB Automatic Deployment of Peer-to-Peer Network Infrastructures