Software Defined Networking for big-data science Eric Pouyoul Chin - PowerPoint PPT Presentation

Software Defined Networking for big-data science Eric Pouyoul Chin Guok Inder Monga (presenting) TERENA Network Architects meeting, Copenhagen November 21 st , 2012

ESnet: World’s Leading Science Network ASIA-PACIFIC RUSSIA US R&E CANADA (ASGC/Kreonet2/ AND CHINA CANADA FRANCE CERN (DREN/Internet2/NLR) (CANARIE) TWAREN) (GLORIAD) (CANARIE) LHCONE (OpenTransit) (USLHCNet) RUSSIA AND CHINA (GLORIAD) ASIA-PACIFIC (KAREN/KREONET2/ NUS-GP/ODN/ SEATTLE REANNZ/SINET/ TRANSPAC/TWAREN) PNNL ASIA-PACIFIC (BNP/HEPNET) AUSTRALIA BOSTON US R&E (AARnet) BOISE (Internet2/ NLR) US R&E (DREN/Internet2/ CERN NISN/NLR) LATIN AMERICA SACRAMENTO CLARA/CUDI CHICAGO BNL NEW YORK CANADA US R&E (CANARIE) FNAL LBNL SUNNYVALE AMES PPPL (DREN/Internet2/ ASIA-PACIFIC ANL NASA) EUROPE (ASCC/KAREN/ (GÉANT/ KREONET2/NUS-GP/ WASHINGTON DC US R&E ODN/REANNZ/ NORDUNET) (NASA/NISN/ SINET/TRANSPAC) USDOI) KANSAS CITY SLAC ASIA-PACIFIC DENVER JLAB (SINET) ORNL NASHVILLE ALBUQUERQUE AUSTRALIA EUROPE (AARnet) (GÉANT) LATIN AMERICA ATLANTA (AMPATH/CLARA) El PASO US R&E LATIN AMERICA 100G IP Hubs (DREN/Internet2/ (CLARA/CUDI) NISN) 4x10G IP Hub HOUSTON Major R&E and International peering connections Lawrence Berkeley National Laboratory U.S. Department of Energy | Office of Science

Problems = Opportunities for innovation (1) Elephant Flows: ‘big-data’ movement for Science, end-to-end Lawrence Berkeley National Laboratory U.S. Department of Energy | Office of Science

Complexity = Opportunity (2): Global Multi-Domain Collaborations like LHC detector 1 PB/s O(1-10) meter Level 1 and 2 triggers O(10-100) meters mile CERN → T1 kms s Level 3 trigger LHC Tier 0 O(1) km France 350 565 Deep archive and send Italy 570 920 CERN Computer Center data to Tier 1 centers UK 625 1000 ~50 Gb/s (25Gb/s ATLAS, 25Gb/s CMS) Netherlands 625 1000 500-10,000 km Germany 700 1185 Universities/ Universities/ physics physics Spain 850 1400 Universities/ LHC Tier 1 Data groups groups physics Universities/ Centers Nordic 1300 2100 physics groups Universities/ groups USA – New York 3900 6300 physics groups USA - Chicago 4400 7100 Universities/ physics Canada – BC 5200 8400 Universities/ groups physics Taiwan 6100 9850 groups Universities/ physics groups Universities/ The LHC Open physics Network groups Universities/ Environment physics ( LHCONE ) groups Universities/ physics Universities/ groups physics groups Universities/ Universities/ The LHC Optical physics physics Source: Bill LHC Tier 2 Analysis Universities/ groups Private Network groups Universities/ Universities/ Centers Johnston physics ( LHCOPN ) physics physics groups groups groups

Science DMZ, perfSONAR, NSI, OSCARS, 100G network is the current reality – still a lot of work to be done The following slides are forward-looking perspective Lawrence Berkeley National Laboratory U.S. Department of Energy | Office of Science

Software-Defined Networking Lawrence Berkeley National Laboratory U.S. Department of Energy | Office of Science

What is Software-Defined Networking? (as defined by Scott Shenker, October 2011) http://opennetsummit.org/talks/shenker-tue.pdf “The ability to master complexity is not the same as the ability to extract simplicity” “Abstractions key to extracting simplicity” “SDN is defined precisely by these three abstractions • Distribution, forwarding, configuration “ Lawrence Berkeley National Laboratory U.S. Department of Energy | Office of Science

Fundamental Network Abstraction: a end-to-end circuit Wavelength, PPP, MPLS, L2TP, GRE, NSI-CS… A Z Switching points, store and forward, transformation … Simple, Point-to-point, Provisonable Lawrence Berkeley National Laboratory U.S. Department of Energy | Office of Science

New Network Abstraction: “WAN Virtual Switch” WAN Virtual Switch WAN Virtual Switch Simple, Multipoint, Programmable Configuration abstraction: • Expresses desired behavior • Hides implementation on physical infrastructure It is not only about the concept, but implementation Lawrence Berkeley National Laboratory U.S. Department of Energy | Office of Science

Simple Example: One Virtual Switch per Collaboration ALCF WAN Virtual Switch WAN Virtual Switch OLCF NERSC ASIA-PACIFIC RUSSIA US R&E CANADA (ASGC/Kreonet2/ AND CHINA CANADA FRANCE CERN (DREN/Internet2/NLR) (CANARIE) TWAREN) (GLORIAD) (CANARIE) LHCONE (OpenTransit) (USLHCNet) RUSSIA AND CHINA (GLORIAD) ASIA-PACIFIC (KAREN/KREONET2/ NUS-GP/ODN/ SEATTLE REANNZ/SINET/ TRANSPAC/TWAREN) PNNL ASIA-PACIFIC (BNP/HEPNET) AUSTRALIA BOSTON US R&E (AARnet) BOISE (Internet2/ NLR) US R&E (DREN/Internet2/ CERN NISN/NLR) LATIN AMERICA SACRAMENTO CLARA/CUDI CHICAGO BNL NEW YORK CANADA US R&E (CANARIE) LBNL AMES FNAL PPPL SUNNYVALE (DREN/Internet2/ ASIA-PACIFIC ANL NASA) (ASCC/KAREN/ EUROPE KREONET2/NUS-GP/ (GÉANT/ WASHINGTON DC US R&E ODN/REANNZ/ NORDUNET) (NASA/NISN/ SINET/TRANSPAC) USDOI) KANSAS CITY SLAC ASIA-PACIFIC DENVER JLAB (SINET) ORNL NASHVILLE ALBUQUERQUE AUSTRALIA EUROPE (AARnet) (GÉANT) LATIN AMERICA ATLANTA (AMPATH/CLARA) El PASO LATIN AMERICA US R&E (DREN/Internet2/ (CLARA/CUDI) NISN) HOUSTON Lawrence Berkeley National Laboratory U.S. Department of Energy | Office of Science

Programmability Site Domain OpenFlow Controller OF protocol WAN Virtual Switch WAN Virtual Switch WAN Domain Expose ‘flow’ programming interface leveraging standard OF protocol Lawrence Berkeley National Laboratory U.S. Department of Energy | Office of Science

“Programmable” by end-sites Program flows: Science Flow1: Science Flow2: Science Flow3: WAN Virtual Switch WAN Virtual Switch App App 1 2 App App OF Ctrl. 1 2 OF Ctrl. Multi-Domain Wide Area Network Lawrence Berkeley National Laboratory U.S. Department of Energy | Office of Science

Many collaborations, Many Virtual Switches WAN Virtual Switch WAN Virtual Switch WAN Virtual Switch WAN Virtual Switch WAN Virtual Switch WAN Virtual Switch WAN Virtual Switch WAN Virtual Switch WAN Virtual Switch WAN Virtual Switch Multi-Domain Wide Area Network Lawrence Berkeley National Laboratory U.S. Department of Energy | Office of Science

SRS Demonstration Physical Topology Ciena 5410 @BNL @Ciena booth @ANL DTNs SRS Brocade @SCinet OSCARS virtual circuits NEC IP8800 @ LBL DTNs: Data Transfer Nodes Lawrence Berkeley National Laboratory U.S. Department of Energy | Office of Science

Virtual Switch Implementation: Mapping abstract model to the physical Create Virtual switch: • Specify edge OF ports • Specify backplane topology and SRS Virtual Switch SRS Virtual Switch bandwidth • Policy constraints like flowspace • Store the switch into a topology service A B C D Virtual Physical C B OF Switch OF Switch A OF Switch OF Switch D Lawrence Berkeley National Laboratory U.S. Department of Energy | Office of Science

WAN Virtual Switch: Deploying it as a service App 1 App 2 Policy/Isolation of OF End-site customer OF Customer Flowvisor OF controller control OF OpenFlow API Virtual Switch Virtual Switch Application Software stack Virtual SW OSCARS Controller Client Infrastructure OF OSCARS API Software, Slicing OSCARS Network Flowvisor and provisioning OF OF Switch Wide Area Network OF Switch OF Switch OF Switch Lawrence Berkeley National Laboratory U.S. Department of Energy | Office of Science

Example of ping across WAN virtual switch 3. Need to ARP 6. virtual-mac-addresses learned and mapped to real flow SRS Virtual Switch SRS Virtual Switch 4. ARP 2. Packet_in H2 5. ARP response 4. ARP 1. Ping H2 OF Switch H1 7. flow_mod 8. Ping H2 OF Switch 8. Ping H2 OF Switch Lawrence Berkeley National Laboratory U.S. Department of Energy | Office of Science

What does this mean for networking? User OpenFlow SDN Controller Customer/User Control Plane Policy and Isolation Multi-domain Network Network Network Network WAN Virtual Switch Service Service Interface Interface Interface Interface Programmable service provisioning plane Legacy and OpenFlow control plane End-to-End Dataplane • Creation of a programmable network provisioning layer • Sits on top of the “network OS” Lawrence Berkeley National Laboratory U.S. Department of Energy | Office of Science

Summary • Powerful network abstraction • Files / Storage • Benefits • Simplicity for the end-site • Works with off-the-shelf, open-source controller • Topology simplification • Generic code for the network provider • Virtual switch can be layered over optical, routed or switched network elements • OpenFlow support needed on edge devices only, core stays same • Programmability for applications • Allows end-sites to innovate and use the WAN effectively Lawrence Berkeley National Laboratory U.S. Department of Energy | Office of Science