DYNES: DYnamic NEtwork System Artur Barczyk California Institute of Technology / US LHCNet TERENA e2e Workshop TERENA e2e Workshop Prague, November 29 th , 2010 1
2 Deployment Plan What is DYNES OUTLINE Status
DYNES Overview What is DYNES? • – A U.S-wide dynamic network “cyber-instrument” spanning ~40 US universities and ~14 Internet2 connectors – Extends Internet2’s dynamic network service “ION” into U.S. regional networks and campuses; Aims to support LHC traffic (also internationally) – Based on the implementation of the Inter-Domain Circuit protocol developed by ESnet and Internet2; Cooperative development also with GEANT, GLIF Who is it? • – Collaborative team: Internet2, Caltech, Univ. of Michigan, Vanderbilt – The LHC experiments, astrophysics community, WLCG, OSG, other VOs – The community of US regional networks and campuses What are the goals? • – Support large, long-distance scientific data flows in the LHC, other programs (e.g. LIGO, Virtual Observatory), & the broader scientific community – Build a distributed virtual instrument at sites of interest to the LHC but available to R&E community generally 3
DYNES Team Internet2, • Caltech, V Vanderbilt, d bilt Univ. of Michigan PI E i PI: Eric Boyd B d • (Internet2) Co-PIs: • – Harvey Newman (Caltech) – Paul Sheldon (V (Vanderbilt) d bilt) – Shawn McKee (Univ. of Michigan) Michigan) http://www.internet2.edu/dynes 4
The Problem to be Addressed Sustained throughputs at 1-10 Gbps (and some > 10 Gbps) are in • production use today by some Tier2s as well as Tier1s LHC data volumes and transfer rates are expected to expand by an order • of magnitude over the next several years – As higher capacity storage and regional, national and transoceanic 40G and 100 Gbps network links become available and affordable. 40G d 100 Gb t k li k b il bl d ff d bl Network usage on this scale can only be accommodated with planning, • an appropriate architecture, and nationwide and Int’l community involvement by i l t b – The LHC groups at universities and labs – Campuses, regional and state networks connecting to Internet2 – ESnet, US LHCNet, NSF/IRNC, other major networks in US & Europe Network resource allocation and data operations need to be consistent • – DYNES will help provide standard services and low cost equipment – DYNES will help provide standard services and low cost equipment to help meet the needs 5
Addressing the The Problem with Dynamic Network Circuits t y a c et o C cu ts DYNES will deliver the needed capabilities to the LHC, and to the broader • scientific community at all the campuses served, by coupling to their analysis systems: l i t – Dynamic network circuit provisioning: IDC Controller – Data transport: Low Cost IDC-capable Ethernet Switch; FDT Server for high th throughput, Low cost storage array where needed (also non-LHC) h t L t t h d d ( l LHC) – End-to-end monitoring services DYNES does not fund more bandwidth, but provides access to Internet2’s • dynamic circuit network (“ION”), plus the standard mechanisms, tools and equipment needed – To build circuits with bandwidth guarantees across multiple network domains, across the U.S. and to Europe in the future th U S d t E i th f t • In a manageable way, with fair-sharing • Will require scheduling services at some stage – To build a community with high throughput capability, using standardized, T b ild i i h hi h h h bili i d di d common methods 6
Leveraging on the Internet2 Dynamic Circuit Network y a c C cu t et o 7
DYNES System Description AIM: extend hybrid & dynamic capabilities to campus & regional networks. • – A DYNES instrument must provide two basic capabilities at the Tier 2S, Tier3s and regional networks: and regional networks: 1. Network resource allocation such as bandwidth to ensure transfer performance 2. Monitoring of the network and data transfer 2 M it i f th t k d d t t f performance • All networks in the path require the ability to allocate network resources and monitor to allocate network resources and monitor the transfer. This capability currently exists on backbone networks such as Internet2 and ESnet but is not widespread at the campus ESnet, but is not widespread at the campus and regional level. – In addition Tier 2 & 3 sites require: Two typical transfers that DYNES 3 Hardware at the end sites capable of making 3. Hardware at the end sites capable of making supports: one Tier2 - Tier3 and supports: one Tier2 - Tier3 and another Tier1-Tier2. optimal use of the available network resources The clouds represent the network domains involved in such a transfer . 8
DYNES: Regional Network - Instrument Design st u e t es g Regional networks require • 1. An Ethernet switch 2. An Inter-domain Controller (IDC) The configuration of the IDC • consists of OSCARS, DRAGON, and perfSONAR. This allows the regional network to provision resources on-demand through i t interaction with the other ti ith th th instruments At the network level, each regional connects the incoming campus connection to the Ethernet switch provided. A regional network does not • Optionally, if a regional network already has a qualified switch require a disk array or FDT server require a disk array or FDT server compatible with the dynamic software that they prefer they compatible with the dynamic software that they prefer, they may use that instead, or in addition to the provided because they are providing equipment. The Ethernet switch provides a VLAN dynamically transport for the Tier 2 and Tier 3 allocated by OSCARS & DRAGON. The VLAN has quality of service (QoS) parameters set to guarantee the bandwidth data transfers not initiating them data transfers, not initiating them. requirements of the connection as defined in the VLAN These requirements of the connection as defined in the VLAN. These parameters are determined by the original circuit request from the researcher / application. through this VLAN, the regional provides transit between the campus IDCs connected in the same region or to the global IDC infrastructure. 9
DYNES: Tier2 and Tier3 Instrument Design st u e t es g Each DYNES (sub-)instrument • at a Tier2 or Tier3 site consists of the following hardware of the following hardware, combining low cost & high performance: 1 An Inter-domain Controller (IDC) 1. An Inter-domain Controller (IDC) 2. An Ethernet switch 3. A Fast Data Transfer (FDT) server Sites with 10GE server. Sites with 10GE throughput capability will have a dual-port Myricom 10GE The Fast Data Transfer (FDT) server connects to the disk array via the SAS controller and runs FDT software developed by Caltech. network interface in the server. network interface in the server. FDT i FDT is an asynchronous multithreaded system that automatically h l i h d d h i ll 4. An optional attached disk array adjusts I/O and network buffers to achieve maximum network utilization. The disk array stores datasets to be transferred among with a Serial Attached SCSI the sites in some cases. The FDT server serves as an aggregator/ throughput optimizer in this case, feeding smooth flows over the (SAS) controller capable of ( ) p networks directly to the Tier2 or Tier3 clusters. The IDC server networks directly to the Tier2 or Tier3 clusters The IDC server several hundred MBytes/sec to handles the allocation of network resources on the switch, inter- actions with other DYNES instruments related to network pro- local storage. visioning, and network performance monitoring. The IDC creates virtual LANs (VLANs) as needed. 10
Local Connectivity Options DYNES offers several connectivity options for the local sites and the RONs. Two examples: Campus using 1 of 2 connections to the Connector and campus incorporating regional for DYNES and the other for DYNES as part of production general purpose IP connectivity infrastructure 11
Fast Data Transfer (FDT) • DYNES instrument includes a storage element, with FDT as data transfer application pp • FDT is an open source application for efficient data transfers • Easy to use: similar syntax with SCP, iperf/netperf • Written in java and runs on all major platforms W itt i j d ll j l tf • Single .jar file (~800 KB) • Based on an asynchronous, multithreaded system Based on an asynchronous, multithreaded system • Uses the New I/O (NIO) interface and is able to: – stream continuously a list of files – use independent threads to read and write on each physical device i d d t th d t d d it h h i l d i – transfer data in parallel on multiple TCP streams, when necessary – use appropriate size of buffers for disk IO and networking – resume a file transfer session 12
Recommend
More recommend
