CS455: Introduction to Distributed Systems [Spring 2020] Dept. Of Computer Science , Colorado State University CS 455: I NTRODUCTION T O D ISTRIBUTED S YSTEMS [ A RCHITECTURES /T OPOLOGY ] Decentralized topologies Is the set of nodes a network? No, it’s all about how you connect them Giving each, its own quirk And properties that make the system hum ¨ Shrideep Pallickara This you probably knew, ¨ Computer Science Your networks tell a lot about you ¨ Colorado State University CS455: Introduction to Distributed Systems C OM TER S CI NCE D EPAR OMPUTE CIENCE EPARTMEN ENT http: ht p://www.cs. cs.co colost state.edu/~cs4 cs455 Topics covered in this lecture ¨ Decentralized architectures ¨ Topologies ¤ Regular graphs ¤ Random graphs ¤ Small world graphs ¤ Power law networks Ryan Stern and Shrideep Pallickara. On the Role of Topology in Autonomously Coping with Failures in Content Dissemination Systems. Proceedings of the ACM Cloud and Autonomic Computing Conference . Miami, USA. 2013. Professor: S HRIDEEP P ALLICKARA CS455: Introduction to Distributed Systems C OM TER S CI NCE D EPAR OMPUTE CIENCE EPARTMEN ENT ht http: p://www.cs. cs.co colost state.edu/~cs4 cs455 L26.1 S LIDES C REATED B Y : S HRIDEEP P ALLICKARA
CS455: Introduction to Distributed Systems [Spring 2020] Dept. Of Computer Science , Colorado State University D ECENTRALIZED A RCHITECTURES CS455: Introduction to Distributed Systems C OM TER S CI NCE D EPAR OMPUTE CIENCE EPARTMEN ENT http: ht p://www.cs. cs.co colost state.edu/~cs4 cs455 Decentralized architectures ¨ Server may be split up into logically equivalent parts ¤ Each part operates on its share of the dataset ¤ Balance the load ¨ Interaction between processes is symmetric ¤ Each peer acts as a client and a server Professor: S HRIDEEP P ALLICKARA CS455: Introduction to Distributed Systems C OM TER S CI NCE D EPAR OMPUTE CIENCE EPARTMEN ENT ht http: p://www.cs. cs.co colost state.edu/~cs4 cs455 L26.2 S LIDES C REATED B Y : S HRIDEEP P ALLICKARA
CS455: Introduction to Distributed Systems [Spring 2020] Dept. Of Computer Science , Colorado State University Structured Peer to Peer Architectures: Distributed hash tables ¨ Data items are assigned an identifier from a large random space § 128-bit UUIDs or 160-bit SHA-1 digests ¨ Nodes are also assigned a number from the same identifier space Professor: S HRIDEEP P ALLICKARA CS455: Introduction to Distributed Systems C OM TER S CI NCE D EPAR OMPUTE CIENCE EPARTMEN ENT http: ht p://www.cs. cs.co colost state.edu/~cs4 cs455 Crux of the DHT problem ¨ Implement an efficient, deterministic scheme to map data item to node ¨ When you look up a data item? ¤ Network address of node holding the data is returned Professor: S HRIDEEP P ALLICKARA CS455: Introduction to Distributed Systems C OM TER S CI NCE D EPAR OMPUTE CIENCE EPARTMEN ENT ht http: p://www.cs. cs.co colost state.edu/~cs4 cs455 L26.3 S LIDES C REATED B Y : S HRIDEEP P ALLICKARA
CS455: Introduction to Distributed Systems [Spring 2020] Dept. Of Computer Science , Colorado State University A quick look at the Chord system ¨ Nodes are organized into a ring ¨ Data item with key k is mapped to a node with the smallest id ≥ k ¤ Also referred to as successor( k ) Professor: S HRIDEEP P ALLICKARA CS455: Introduction to Distributed Systems C OM TER S CI NCE D EPAR OMPUTE CIENCE EPARTMEN ENT ht http: p://www.cs. cs.co colost state.edu/~cs4 cs455 Mapping of data items to nodes in Chord Actual Node 0 1 15 2 {0,1} 14 {13,14,15} 3 13 Associated data keys {2,3,4} 4 {8,9,10,11,12} 12 5 11 {5,6,7} 6 10 7 9 8 Professor: S HRIDEEP P ALLICKARA CS455: Introduction to Distributed Systems C OM TER S CI NCE D EPAR OMPUTE CIENCE EPARTMEN ENT ht http: p://www.cs. cs.co colost state.edu/~cs4 cs455 L26.4 S LIDES C REATED B Y : S HRIDEEP P ALLICKARA
CS455: Introduction to Distributed Systems [Spring 2020] Dept. Of Computer Science , Colorado State University Chord lookup example for k=54 N 1 lookup(54) K54 N N 8 56 N 1/16 51 N 14 N 47 1/8 1/2 N 21 N 42 N 38 Professor: S HRIDEEP P ALLICKARA CS455: Introduction to Distributed Systems C OM TER S CI NCE D EPAR OMPUTE CIENCE EPARTMEN ENT ht http: p://www.cs. cs.co colost state.edu/~cs4 cs455 When a node wants to join? ¨ Generate a random id ¤ Probability of collisions is low ¨ lookup(id) ¤ Will return successor(id) ¨ Contact successor(id) and its predecessor ¤ Insert self in the ring ¤ Transfer data items Professor: S HRIDEEP P ALLICKARA CS455: Introduction to Distributed Systems C OM TER S CI NCE D EPAR OMPUTE CIENCE EPARTMEN ENT ht http: p://www.cs. cs.co colost state.edu/~cs4 cs455 L26.5 S LIDES C REATED B Y : S HRIDEEP P ALLICKARA
CS455: Introduction to Distributed Systems [Spring 2020] Dept. Of Computer Science , Colorado State University An example of inserting a new node Actual Node 0 1 15 2 {0,1} 14 {13,14,15} 3 13 Associated data keys {2,3,4} 4 {8,9,10,11,12} 12 5 11 {5,6,7} 6 10 New node 10 7 will be inserted 9 8 Professor: S HRIDEEP P ALLICKARA CS455: Introduction to Distributed Systems C OM TER S CI NCE D EPAR OMPUTE CIENCE EPARTMEN ENT http: ht p://www.cs. cs.co colost state.edu/~cs4 cs455 An example of inserting a new node Actual Node 0 1 15 2 {0,1} 14 {13,14,15} 3 13 Associated data keys {2,3,4} 4 {11,12} 12 5 11 {8,9,10} {5,6,7} 6 10 7 9 8 Professor: S HRIDEEP P ALLICKARA CS455: Introduction to Distributed Systems C OM TER S CI NCE D EPAR OMPUTE CIENCE EPARTMEN ENT ht http: p://www.cs. cs.co colost state.edu/~cs4 cs455 L26.6 S LIDES C REATED B Y : S HRIDEEP P ALLICKARA
CS455: Introduction to Distributed Systems [Spring 2020] Dept. Of Computer Science , Colorado State University Unstructured P2P networks that rely on random graphs ¨ Maintain connections to randomly chosen live nodes ¨ To locate a data item ¤ Flood the network Professor: S HRIDEEP P ALLICKARA CS455: Introduction to Distributed Systems C OM TER S CI NCE D EPAR OMPUTE CIENCE EPARTMEN ENT ht http: p://www.cs. cs.co colost state.edu/~cs4 cs455 Hierarchical organization of nodes Super peer Regular peer Professor: S HRIDEEP P ALLICKARA CS455: Introduction to Distributed Systems C OM TER S CI NCE D EPAR OMPUTE CIENCE EPARTMEN ENT ht http: p://www.cs. cs.co colost state.edu/~cs4 cs455 L26.7 S LIDES C REATED B Y : S HRIDEEP P ALLICKARA
CS455: Introduction to Distributed Systems [Spring 2020] Dept. Of Computer Science , Colorado State University Superpeer networks ¨ The client-superpeer relationship is fixed ¤ When a peer joins, it attaches itself to the superpeer and stays attached till it leaves ¨ Superpeers are expected to be long-lived processes with high- availability ¨ Selecting nodes that are eligible to be superpeers? ¤ Closely related to the leader election problem Professor: S HRIDEEP P ALLICKARA CS455: Introduction to Distributed Systems C OM TER S CI NCE D EPAR OMPUTE CIENCE EPARTMEN ENT http: ht p://www.cs. cs.co colost state.edu/~cs4 cs455 S MALL W ORLDS CS455: Introduction to Distributed Systems C OM TER S CI NCE D EPAR OMPUTE CIENCE EPARTMEN ENT ht http: p://www.cs. cs.co colost state.edu/~cs4 cs455 L26.8 S LIDES C REATED B Y : S HRIDEEP P ALLICKARA
CS455: Introduction to Distributed Systems [Spring 2020] Dept. Of Computer Science , Colorado State University Stanley Milgram’s experiment on social networks ¨ In 1967 he mailed 160 letters ¨ People were randomly chosen from Omaha, Nebraska ¨ Objective was to pass their letter ¤ T ARGET : Stock broker in Boston, MA ¤ C ONSTRAINT : Use intermediary known to them on a first-name basis Professor: S HRIDEEP P ALLICKARA CS455: Introduction to Distributed Systems C OM TER S CI NCE D EPAR OMPUTE CIENCE EPARTMEN ENT ht http: p://www.cs. cs.co colost state.edu/~cs4 cs455 Results: It’s a small, small world ¨ 42 letters made it through ¤ Median was just 5.5. intermediaries ¤ US Population in 1967: 200 million ¨ First demonstration of what is known as the small world effect Professor: S HRIDEEP P ALLICKARA CS455: Introduction to Distributed Systems C OM TER S CI NCE D EPAR OMPUTE CIENCE EPARTMEN ENT ht http: p://www.cs. cs.co colost state.edu/~cs4 cs455 L26.9 S LIDES C REATED B Y : S HRIDEEP P ALLICKARA
CS455: Introduction to Distributed Systems [Spring 2020] Dept. Of Computer Science , Colorado State University Intuitively it seems that the pathlengths should have been much higher ¨ People’s social circle is cliquish or clustered ¨ People you know, know each other Professor: S HRIDEEP P ALLICKARA CS455: Introduction to Distributed Systems C OM TER S CI NCE D EPAR OMPUTE CIENCE EPARTMEN ENT ht http: p://www.cs. cs.co colost state.edu/~cs4 cs455 The key is the distribution of links within social networks ¨ Some acquaintances are relatively isolated ¨ Some have wide ranging connections ¤ Play a critical role in bringing network closer together ¨ Milgram experiment ¤ ¼ of the successful chains passed through a local storekeeper Professor: S HRIDEEP P ALLICKARA CS455: Introduction to Distributed Systems C OM TER S CI NCE D EPAR OMPUTE CIENCE EPARTMEN ENT ht http: p://www.cs. cs.co colost state.edu/~cs4 cs455 L26.10 S LIDES C REATED B Y : S HRIDEEP P ALLICKARA
Recommend
More recommend
