p2p content distribution bittorrent and spotify
play

P2P Content Distribution BitTorrent and Spotify Amir H. Payberah - PowerPoint PPT Presentation

May 27, 2023 •242 likes •1.07k views

P2P Content Distribution BitTorrent and Spotify Amir H. Payberah amir@sics.se Amirkabir University of Technology (Tehran Polytechnic) Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 1 / 62 Amir H. Payberah (Tehran

  1. P2P Content Distribution BitTorrent and Spotify Amir H. Payberah amir@sics.se Amirkabir University of Technology (Tehran Polytechnic) Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 1 / 62

  2. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 2 / 62

  3. Possible Solutions for Content Distribution Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 3 / 62

  4. Client-Server Model Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 4 / 62

  5. Client-Server Model Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 5 / 62

  6. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 6 / 62

  7. The Client-Server Model Problems ◮ Scalability? Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 7 / 62

  8. The Client-Server Model Problems ◮ Scalability? ◮ Single Point of failure? Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 7 / 62

  9. The Client-Server Model Problems ◮ Scalability? ◮ Single Point of failure? Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 7 / 62

  10. Client-Server Systems Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 8 / 62

  11. The Client-Server Model Problem Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 9 / 62

  12. Scalable and Fault-Tolerant Client-Server Model Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 10 / 62

  13. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 11 / 62

  14. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 12 / 62

  15. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 13 / 62

  16. Peer-to-Peer Model Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 14 / 62

  17. Peer-to-Peer (P2P) Model Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 15 / 62

  18. P2P Challenges ◮ Churn in the system ◮ Free-riding problem ◮ Bottleneck in the overlay network ◮ Connectivity problem, e.g., NAT Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 16 / 62

  19. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 17 / 62

  20. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 18 / 62

  21. How To Discover Data? Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 19 / 62

  22. Possible Solutions - First Generation ◮ Central directory Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 20 / 62

  23. Possible Solutions - Second Generation ◮ Flooding Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 21 / 62

  24. Possible Solutions - Third Generation ◮ Distributed Hash Table (DHT) Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 22 / 62

  25. P2P Content Distribution Applications - File Sharing Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 23 / 62

  26. P2P Content Distribution Applications - Media Streaming Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 24 / 62

  27. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 25 / 62

  28. BitTorrent ◮ BitTorrent is a system for efficient and scalable replication of large amounts of static data. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 26 / 62

  29. BitTorrent Players Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 27 / 62

  30. Files ◮ Files are broken into pieces of size between 64KB and 1MB. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 28 / 62

  31. .torrent Files ◮ Metadata ◮ Contains: • URL of tracker • Information about the file, e.g., filename, length, ... Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 29 / 62

  32. The Core Idea ◮ A peer obtains .torrent file. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 30 / 62

  33. The Core Idea ◮ A peer obtains .torrent file. ◮ It, then, connects to the tracker. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 30 / 62

  34. The Core Idea ◮ A peer obtains .torrent file. ◮ It, then, connects to the tracker. ◮ The tracker tells the peers from which other peers to download the pieces of the file. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 30 / 62

  35. The Core Idea ◮ A peer obtains .torrent file. ◮ It, then, connects to the tracker. ◮ The tracker tells the peers from which other peers to download the pieces of the file. ◮ Peers use this information to communicate with each other. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 30 / 62

  36. The Core Idea ◮ A peer obtains .torrent file. ◮ It, then, connects to the tracker. ◮ The tracker tells the peers from which other peers to download the pieces of the file. ◮ Peers use this information to communicate with each other. ◮ The peers send information about the file and themselves to tracker. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 30 / 62

  37. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 31 / 62

  38. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 32 / 62

  39. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 33 / 62

  40. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 34 / 62

  41. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 35 / 62

  42. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 36 / 62

  43. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 37 / 62

  44. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 38 / 62

  45. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 39 / 62

  46. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 40 / 62

  47. What About Free Riders? Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 41 / 62

  48. Tit-For-Tat Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 42 / 62

  49. Question ◮ From which peers download the pieces? Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 43 / 62

  50. Peer Selection ◮ Use choking algorithm to choose peer to download pieces. ◮ Decision to choke/unchoke based on tit-for-tat. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 44 / 62

  51. Discover More Cooperating Peers ◮ Optimistic unchoking ◮ Allocate an upload slot to a randomly chosen uncooperative peer Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 45 / 62

  52. Snubbed Peers ◮ If all its peers choke it. ◮ Increase the number of optimistic unchokes. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 46 / 62

  53. Question ◮ Which piece? Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 47 / 62

  54. Piece Selection ◮ Rarest first: common parts left for later Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 48 / 62

  55. Piece Selection ◮ Rarest first: common parts left for later ◮ Random first piece: start-up need to get a complete piece Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 48 / 62

  56. Piece Selection ◮ Rarest first: common parts left for later ◮ Random first piece: start-up need to get a complete piece ◮ Endgame mode: broadcast for all remaining blocks Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 48 / 62

  57. BitTorrent Extension ◮ Distributed tracker ◮ Peer-exchange Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 49 / 62

  58. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 50 / 62

  59. Spotify ◮ Active users: over 50 million ◮ Number of songs: over 20 million ◮ Number of songs added per day: over 20000 ◮ Number of playlists: over 1.5 billion created so far ◮ Available in 58 countries ◮ Legal Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 51 / 62

  60. The Core Idea ◮ Request first piece from Spotify servers. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 52 / 62

  61. The Core Idea ◮ Request first piece from Spotify servers. ◮ Meanwhile, search P2P network for remainder. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 52 / 62

  62. The Core Idea ◮ Request first piece from Spotify servers. ◮ Meanwhile, search P2P network for remainder. ◮ Switch back and forth between Spotify servers and peers as needed. Amir H. Payberah (Tehran Polytechnic) P2P Content Distribution 1393/8/27 52 / 62

Recommend

Comparison of BitTorrent with Traditional Content Distribution Networks (CDNs) ENSC 835 High

Comparison of BitTorrent with Traditional Content Distribution Networks (CDNs) ENSC 835 High

Comparison of BitTorrent with Traditional Content Distribution Networks (CDNs) ENSC 835 High Performance Networks Catherine Chan Sean Puttergill Roadmap Introduction & Motivation Overview of Related Work Technical

374 views • 23 slides
Scaling Data Infrastructure @ Spotify matti@spotify.com kalvans@spotify.com Mrti Kalvns

Scaling Data Infrastructure @ Spotify matti@spotify.com kalvans@spotify.com Mrti Kalvns

Scaling Data Infrastructure @ Spotify matti@spotify.com kalvans@spotify.com Mrti Kalvns Matti Pehrs kalvans@spotify.com matti@spotify.com Agenda 1. Data at Spotify 2. Summer of 2015 3. Challenges & Victory Datamon

543 views • 42 slides
BitTorrent Mads Dar Kristensen Niels Olof Bouvin 1 Overview BitTorrent terms The BitTorrent

BitTorrent Mads Dar Kristensen Niels Olof Bouvin 1 Overview BitTorrent terms The BitTorrent

BitTorrent Mads Dar Kristensen Niels Olof Bouvin 1 Overview BitTorrent terms The BitTorrent protocol The life of a torrent Attacking BitTorrent 2 BitTorrent terms The BitTorrent protocol operates with these important terms: Tracker: a

1k views • 52 slides
The Evolution of Hadoop at Spotify Rafal Wojdyla (rav@spotify.com) Josh Baer (jbx@spotify.com)

The Evolution of Hadoop at Spotify Rafal Wojdyla (rav@spotify.com) Josh Baer (jbx@spotify.com)

The Evolution of Hadoop at Spotify Rafal Wojdyla (rav@spotify.com) Josh Baer (jbx@spotify.com) @l_phant @ravwojdyla Technical Product Owner Data Engineer Data Infrastructure Hadoop Team Overview Growing Pains Gaining Focus The

945 views • 62 slides
Characterization of BitTorrent Swarms and their Distribution in the Internet Tobias Hofeld ,

Characterization of BitTorrent Swarms and their Distribution in the Internet Tobias Hofeld ,

Institute of Computer Science Chair of Communication Networks Prof. Dr.-Ing. P. Tran-Gia Characterization of BitTorrent Swarms and their Distribution in the Internet Tobias Hofeld , Frank Lehrieder, David Hock, Simon Oechsner University of

416 views • 13 slides
The Evolution of Hadoop at Spotify Rafal Wojdyla (rav@spotify.com) Josh Baer (jbx@spotify.com)

The Evolution of Hadoop at Spotify Rafal Wojdyla (rav@spotify.com) Josh Baer (jbx@spotify.com)

The Evolution of Hadoop at Spotify Rafal Wojdyla (rav@spotify.com) Josh Baer (jbx@spotify.com) @l_phant @ravwojdyla Technical Product Owner Data Engineer Hadoop Squad Hadoop Squad Overview Growing Pains Gaining Focus The

774 views • 51 slides
CompSci 514: Computer Networks Lecture 21-2: From BitTorrent to BitTyrant Problem Statement

CompSci 514: Computer Networks Lecture 21-2: From BitTorrent to BitTyrant Problem Statement

CompSci 514: Computer Networks Lecture 21-2: From BitTorrent to BitTyrant Problem Statement ... Server One-to-many content distribution Millions of clients downloading from the same server Evolving Solutions Observation:

737 views • 35 slides
Connectivity Properties of Mainline BitTorrent DHT Nodes Raul Jimenez, Flutra Osmani, Bjrn

Connectivity Properties of Mainline BitTorrent DHT Nodes Raul Jimenez, Flutra Osmani, Bjrn

Connectivity Properties of Mainline BitTorrent DHT Nodes Raul Jimenez, Flutra Osmani, Bjrn Knutsson KTH, Sweden 1 IEEE P2P'09 9-11 Sept Seattle, WA Background P2P-Next Large EU project Content distribution platform on

472 views • 22 slides
The Spotify Platform WOW Hack Gteborg 2014 Per-Olov Jernberg @possan @SpotifyPlatform Spotify

The Spotify Platform WOW Hack Gteborg 2014 Per-Olov Jernberg @possan @SpotifyPlatform Spotify

The Spotify Platform WOW Hack Gteborg 2014 Per-Olov Jernberg @possan @SpotifyPlatform Spotify APIs, SDKs and Widgets Spotify Widgets Play Button Follow Button Libspotify Embed Spotify in your application or hardware. iOS SDK Beta

326 views • 21 slides
Collusion-resilient credit-based reputation for peer-to-peer content distribution Nguyen Tran,

Collusion-resilient credit-based reputation for peer-to-peer content distribution Nguyen Tran,

Collusion-resilient credit-based reputation for peer-to-peer content distribution Nguyen Tran, Jinyang Li, Lakshminarayanan Subramanian New York University NetEcon10 1 Incentive in P2P CDNs A solved problem? Yes BitTorrent

683 views • 54 slides
Bittorrent Introduction Introduction to BitTorrent Efficiency & Reliability The

Bittorrent Introduction Introduction to BitTorrent Efficiency & Reliability The

Bittorrent Introduction Introduction to BitTorrent Efficiency & Reliability The incentive mechanism Trackerless with DHT Arvid Norberg arvid@cs.umu.se http://libtorrent.net Distributed systems C, TDBC85, Ume University,

304 views • 8 slides
Key Regression Fu, Kamara, Kohno Divya Muthukumaran Content Distribution Content publishers

Key Regression Fu, Kamara, Kohno Divya Muthukumaran Content Distribution Content publishers

Key Regression Fu, Kamara, Kohno Divya Muthukumaran Content Distribution Content publishers -> Content distribution Outsourcing storage and distribution Content distribution Networks: Akamai SpeedEra MirrorImage..

945 views • 24 slides
Danielle de Ferrari Sarah de Ferrari Source: Spotify Source: Spotify, 2014 Source: Mashable,

Danielle de Ferrari Sarah de Ferrari Source: Spotify Source: Spotify, 2014 Source: Mashable,

Bryce Simantel Christine Osazuwa Danielle de Ferrari Sarah de Ferrari Source: Spotify Source: Spotify, 2014 Source: Mashable, 2013 Source: Spotify, 2013 WEAKNESSES STRENGTHS High frequency in audio & visual Supports the

921 views • 27 slides
BitTorrent-based credit systems BitTorrent - reminder tracker: keeps a record of every

BitTorrent-based credit systems BitTorrent - reminder tracker: keeps a record of every

On the sustainability of BitTorrent-based credit systems BitTorrent - reminder tracker: keeps a record of every transaction, peer and swarm private communities: own incentive policies sharing ratio enforcement, credit amount checking

576 views • 13 slides
CS5412: TORRENTS AND TIT-FOR-TAT Lecture VI Ken Birman BitTorrent 2 Today well be

CS5412: TORRENTS AND TIT-FOR-TAT Lecture VI Ken Birman BitTorrent 2 Today well be

CS5412 Spring 2012 (Cloud Computing: Birman) 1 CS5412: TORRENTS AND TIT-FOR-TAT Lecture VI Ken Birman BitTorrent 2 Today well be focusing on BitTorrent The technology really has three aspects A standard tht BitTorrent client

478 views • 44 slides
nT nTorrent: : Bi BitTorrent in in Nam Named Data a Ne Networ orkin ing Spyros

nT nTorrent: : Bi BitTorrent in in Nam Named Data a Ne Networ orkin ing Spyros

nT nTorrent: : Bi BitTorrent in in Nam Named Data a Ne Networ orkin ing Spyros Mastorakis Internet Research Laboratory UCLA Bi BitTorrent in in TCP/IP IP BitTorrent is a popular peer-to-peer file sharing application

439 views • 10 slides
Music Recommendation in Spotify Boxun Zhang About me Data scientist at Spotify Big hype

Music Recommendation in Spotify Boxun Zhang About me Data scientist at Spotify Big hype

Music Recommendation in Spotify Boxun Zhang About me Data scientist at Spotify Big hype nowadays Build models of user behavior Develop algorithms Design A/B tests Ph.D. in CS from TU Delft (NL) Studied user behavior in

799 views • 20 slides
Project Presentations 1 BitTorrent Raylene Yung Nathan Marz BitTorrent protocol File split

Project Presentations 1 BitTorrent Raylene Yung Nathan Marz BitTorrent protocol File split

CS259: Security Analysis of Network Protocols Winter 2008 Project Presentations 1 BitTorrent Raylene Yung Nathan Marz BitTorrent protocol File split into equal-sized segments called pieces Tracker: server that keeps track of

731 views • 38 slides
Breaking the hierarchy How Spotify enables engineer decision making Kristian Lindwall, Spotify

Breaking the hierarchy How Spotify enables engineer decision making Kristian Lindwall, Spotify

Breaking the hierarchy How Spotify enables engineer decision making Kristian Lindwall, Spotify Introduction 3 Section name Our employees are our most valued resource So where does it go wrong? 6 Section name Lets do this! 7

571 views • 44 slides
Demand-Aware Content Distribution Srinivas Shakkottai Texas A&M University Hybrid content

Demand-Aware Content Distribution Srinivas Shakkottai Texas A&M University Hybrid content

Demand-Aware Content Distribution Srinivas Shakkottai Texas A&M University Hybrid content distribution High level idea: Use P2P dissemination to assist traditional client- server methods, e.g., content delivery network (CDN). Key

366 views • 33 slides
Big Data at Spotify Anders Arpteg, Ph D Analytics Machine Learning, Spotify Quickly about me

Big Data at Spotify Anders Arpteg, Ph D Analytics Machine Learning, Spotify Quickly about me

Big Data at Spotify Anders Arpteg, Ph D Analytics Machine Learning, Spotify Quickly about me Quickly about Spotify What is all the data used for? Quickly about Spark Hadoop MR vs Spark Need for (distributed)

357 views • 34 slides
Coded Caching for Content Distribution Urs Niesen MobiHoc 2018 Importance of Content

Coded Caching for Content Distribution Urs Niesen MobiHoc 2018 Importance of Content

Coded Caching for Content Distribution Urs Niesen MobiHoc 2018 Importance of Content Distribution Video on demand is driving network traffic growth Netflix streaming service, Amazon Prime Video, Hulu, Verizon / Comcast on Demand, . . . IP

3.63k views • 164 slides
Native Content Distribution through Off-Path Content Discovery A Proposal for a Downstream

Native Content Distribution through Off-Path Content Discovery A Proposal for a Downstream

Native Content Distribution through Off-Path Content Discovery A Proposal for a Downstream FIB Opportunistic Off-Path Content Discovery in Information-Centric Networks O. Ascigil, V. Sourlas, I. Psaras, G. Pavlou IEEE LANMAN 2016

431 views • 39 slides
TICKETMASTER SPOTIFY We are proposing a new way for music fans to purchase concert tickets by

TICKETMASTER SPOTIFY We are proposing a new way for music fans to purchase concert tickets by

TICKETMASTER SPOTIFY We are proposing a new way for music fans to purchase concert tickets by identifying legitimate fans through Spotify, as a means to counter bots and scalpers. Business Problem Among fans and artists, Ticketmasters

539 views • 17 slides

More recommend