NL-T1 Expectations, findings, and innovation Bas Kreukniet, Sr - PowerPoint PPT Presentation

NL-T1 Expectations, findings, and innovation Bas Kreukniet, Sr Network Specialist at SURF SARA Geneva Workshop 10 Februari 2014

Outline 1. Expectations from NL-T1 grid administrators 2. Findings while connecting to the LHCONE 3. Innovation: Ethernet OAM and NSI

Expectations from NL-T1 administrators Advice from grid administrators NL-T1: • Bulk data with simple applications (grid-FTP). “ Keep it simple ” • Network provisioning from application is considered “complex” • Don’t rush to merge LHCOPN with LHCONE

NL-T1 connected to LHCONE since 20 Jan 2014

BGP routing starts at the connected site. BGP routing for T1’s: Not only a “NREN thing” – it already starts at your organisation! Focus on some BGP topics for connecting sites to LHC networks

LHCOPN connectivity

LHCONE connectivity LHCONE.inet.0: 133 destinations, 133 routes (133 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both LHCONE-RS NL-T1 GÉANT CERN CERN AS1162 AS20965 AS513 AS20641 128.142.0.0/16 *[BGP/170] 1w2d 04:07:47, MED 120, localpref 100 AS path: 20965 20641 513 I > to 62.40.126.161 via xe-4/0/1.2012

Preferred route: LHCOPN, LHCONE or Internet • primary: direct T1 – T1 over LHCOPN • secondary: T1 – T1 over LHCOPN via another T1 • tertiary: LHCONE • quaternary: Internet

Preferred route: route (a)symmetry If everyone makes this choice, all connections are symmetric. But: sites may have different preferences: ISP-A 10 Gb/s 10 Gb/s site 1 site 2 1 Gb/s 100 Gb/s ISP-B

Tie-breaker between LHCOPN and LHCONE CNAF routers at NL-T1 LHCOPN 131.154.128.0/17 *[BGP/170] 14:58:18, localpref 100 AS path: 34878 137 I LHCONE 131.154.128.0/17 *[BGP/170] 1w2d 04:23:40, MED 120, localpref 100 AS path: 20965 137 I

LHCOPN / LHCONE route preference BNL routes at NL-T1 LHCOPN 130.199.185.0/24 *[BGP/170] 2w5d 09:35:43, MED 10, localpref 100 AS path: 513 43 I > to 192.16.166.73 via xe-1/1/0.0 [BGP/170] 3d 01:15:22, MED 51, localpref 100 AS path: 39590 513 43 I > to 109.105.124.17 via xe-2/1/0.0 LHCONE 130.199.185.0/24 *[BGP/170] 1w2d 04:27:43, MED 120, localpref 100 AS path: 20965 293 43 I > to 62.40.126.161 via xe-4/0/1.2012

Problems and concerns regarding BGP • A site advertising his routes has no control who to send a route to. At best they can give hints with BGP communities. • The site receiving a route decides which route to accept and how to accept.

BGP in LHC networks Only As is not enough. More info needed.  community for origin of a route T1/T2? Or even better: -site connected to LHCOPN -site connected to LHCONE - Site connected to both: Type A “prefer LHCONE for this route” Type B “prefer LHCOPN for this route” - Specials: dedicated link between two (T1) sites. (“VPN”or “private link”)

Route Preference Solutions • Idea: Tag routes with two types of communities: • One for origin or source • One for destinations • See also: BGP hinting by Martin Sweeny (Indiana U) • BGP Always-compare-MED always on. We sometimes add metrics on incoming routes.

Operational Issues • We received routes over LHC from a site, but traffic we send is blackholed by that site • The site was still reachable over the Internet. • This happened to us twice recently: on LHCOPN (accidental route redistribution) and LHCONE (incoming IP filter). • Configuration errors will be made (we’re also just human) • LHCOPN Link NL-T1 – TRIUMF link is still unstable • 31 outages last 4 months  Monitoring remains important!

LHCONE and LHCOPN layer 2 monitoring Ethernet OAM monitoring or Layer 2 monitoring NIKHEF, TRIUMF to participate as Measurement Points Looking for T1’s and T2 to participate Advantages: • Layer2 keep-alive, ping and traceroute • Interdomain, intervendor solution • L2 devices can be made visible • unidirectional fibercuts can be signalled

LHCONE Innovation NL-T1 likes to partcipate in NSI for LHCONE NSI experiences so far: • Use-case: Life Science Grid (LSG) in Holland makes use of NSI since autumn 2013. • Freek contributed to standard • Sander wrote NSI client and implemented it for “Cloud Bypassing ” in the Life Science Grid.

Cloud Bypassing on Life Science Grid (LSG) Life Science grid Life Science grid cluster @ TUD cluster @ AMC t u l a f e d d e f a u AMC l t campus campus network network Regular Internet RUG SARA data campus center network network d e f a u default l t Life Science grid Life Science grid cluster @ SURFsara cluster @ RUG Compute clusters at ±10 locations in the Netherlands

Cloud Bypassing on Life Science Grid (LSG) Life Science grid Life Science grid cluster @ TUD cluster @ AMC t l u a f e d d e f a AMC u l t campus campus More specific network network Regular (BGP) Internet More specific (BGP) RUG SARA data campus center network network d e f a u default l t SURFnet More specific dynamic s p e c i f i c M o r e (BGP) lightpaths ( B G P ) Life Science grid Life Science grid cluster @ SURFsara cluster @ RUG Offloading campus networks (some only have 1 Gb/s Internet)

LHCONE Innovation: Offloading is cheaper Internet full routing (incl backup): € 8k – € 10k per month for 10 Gb/s LHCONE or dynamic lightpath: € 2k – 3k per month for 10 Gb/s

Questions? Erik Bas Diederik Sander Farhad Freek Ruiter Kreukniet Vandevenne Boele Davani Dijkstra