AllHandsMee2ng ESnet:NetworkingforScience March10,2010 - PowerPoint PPT Presentation

Open
Science
Grid
 All
Hands
Mee2ng 
 ESnet:
Networking
for
Science 
 March
10,
2010 
 Mike
O’Connor,
Network
Engineer
 Energy
Sciences
Network
 Lawrence
Berkeley
Na>onal
Lab
 Supporting Advanced Scientific Computing Research • Basic Energy Sciences • Biological and Environmental Research • Fusion Energy Sciences • High Energy Physics • Nuclear Physics

ESnet R&E Peerings Late 2009 Japan (SINet) KAREN/REANNZ Australia (AARNet) ODN Japan Telecom Canada (CA*net4 America CA*net4 MREN Taiwan (TANet2) NLR-Packetnet France StarTap Singaren Internet2 GÉANT in Vienna GLORIAD Taiwan (TANet2, Transpac2 (via USLHCNet circuit) Korea (Kreonet2) (Russia, China) ASCGNet) CUDI SINet (Japan) Korea (Kreonet2 Russia (BINP) CERN/LHCOPN PNNL (USLHCnet: GÉANT DOE+CERN funded) - France, Germany, KAREN / REANNZ Transpac2 Italy, UK, etc Internet2 Korea (kreonet2) CA*net4 SINGAREN Japan (SINet) ODN Japan Telecom BNL America INL StarLight IARC Ames DOE MAN LAN LBL (32 A of A) NETL PPPL LLNL ANL NNSA FNAL SLAC NREL JLAB KCP AMPATH NOAA CLARA Yucca NSTEC (S. America) LANL ORNL Bechtel-NV SNLA OSTI Allied Signal DOE-ALB ORAU GA ARM SRS Pantex CUDI IP (S. America) SDN IP router 20G SDN NLR 10G SDN router MAN Lab Link Optical node Peering Link Lab

On-demand Secure Circuits and Advance Reservation System (OSCARS) Overview Path Computation Scheduling • Topology • AAA • Reachability • Availability • Contraints OSCARS Guaranteed Bandwidth Virtual Circuit Services Provisioning • Signaling • Security • Resiliency/Redundancy

OSCARS is in Production Now • OSCARS is currently being used to support production traffic • Operational Virtual Circuit (VC) support – As of 10/2009, there are 26 long-term production VCs instantiated • 21 VCs supporting HEP – LHC T0-T1 (Primary and Backup) – LHC T1-T2 • 3 VCs supporting Climate – GFDL – ESG • 2 VCs supporting Computational Astrophysics – OptiPortal • Short-term dynamic VCs • Between 1/2008 and 10/2009, there were roughly 4600 successful VC reservations – 3000 reservations initiated by BNL using TeraPaths – 900 reservations initiated by FNAL using LambdaStation – 700 reservations initiated using Phoebus • The adoption of OSCARS as an integral part of the ESnet4 network was a core contributor to ESnet winning the Excellence.gov “Excellence in Leveraging Technology” award given by the Industry Advisory Council’s (IAC) Collaboration and Transformation Shared Interest Group (Apr 2009)

Network Mechanisms Underlying OSCARS LSP between ESnet border (PE) routers is determined using topology information from OSPF-TE. Path of LSP is explicitly directed to take SDN network where possible. Layer 3 VC Service: Packets On the SDN all OSCARS traffic is MPLS switched (layer 2.5). matching reservation profile Best-effort IP traffic can IP flow-spec are filtered out use SDN, but under (i.e. policy based routing), normal circumstances it “policed” to reserved does not because the bandwidth, and injected into OSPF cost of SDN is very SDN SDN SDN an LSP. high Layer 2 VC Service: Packets matching RSVP, MPLS, LDP reservation profile VLAN ID enabled on are filtered out (i.e. L2VPN), Sink internal interfaces “policed” to reserved explicit bandwidth, and injected into Label Switched Path an LSP. Source IP IP IP IP Link ESnet bandwidth WAN policer high-priority queue MPLS labels are attached onto packets from Source and standard, placed in separate queue to ensure Ntfy
 Resv
 best-effort guaranteed bandwidth. WBUI
 APIs
 API
 queue OSCARS
 NS
 PSS
 Core
 Regular production (best-effort) Interface queues OSCARS
 traffic queue. PCE
 AAAS
 IDC



Growing Use of Circuits for Science Within ESnet Slide Courtesy of Phil DeMar FNAL
traffic
over
five
years
–
IP
traffic
is
a
mix
of
science
and
business,
the
growing
 “LightPath”
traffic
is
all
science
traffic


Tier 0 to Tier 1 Virtual Circuits ESnet USLHCNet e600chi FNAL primary 1 (10G) e600ams FNAL primary 2 (8G) star-cr1 BNL primary 1 (10G) CHGO AMST BNL primary 2 (8G) Fermi FNAL backup (3G) BNL backup (3G) star-sdn1 Starlight e600gva2 r02lcg aofa-sdn1 LCG BNL NYCY GNVA r01lcg BNL e600nyc e600gva1 aofa-cr2 CERN 32AOFA Approximate Paths Slide courtesy of USLHCNet

Instantaneous BNL Circuit Traffic Slide Courtesy of John Bigrow (BNL)

BNL OSCARS Virtual Circuits 3/10/10


BNL OSCARS Virtual Circuits 3/10/10


BNL OSCARS Virtual Circuits 3/10/10


Distributed Science and Soft Network Errors • Most of today's LANs and WANs are big & complex, difficult to fully understand, constantly changing. • Distributed data transfers traverse these networks as long duration high bandwidth flows. This is in stark contrast to most other forms of Internet traffic, which networks are usually designed to handle. • Collaborations on the leading edge of scientific data distribution are encountering latent network problems, or “soft errors” that adversely impact performance. • The PerfSONAR project is focused on tools and processes to locate and characterize problems that affect performance, so they can be fixed.

PerfSONAR • An open web-services based framework for collecting, managing and sharing network measurements • The framework is being deployed across the science community • Encouraging people to deploy ‘known good’ measurement points near domain boundaries • Using the framework to find & correct soft network failures.