Improving network agility with seamless BGP reconfigurations - PowerPoint PPT Presentation

Improving network agility with seamless BGP reconfigurations Laurent Vanbever vanbever@cs.princeton.edu IRTF Open Meeting, IETF87 July, 30 2013 Based on joint work with Stefano Vissicchio, Luca Cittadini, Cristel Pelsser, Pierre François and Olivier Bonaventure

When you are changing “ the tires of a moving car --Vijay Gill

When you are changing “ the tires of a moving car make sure one wheel is on the ground at all time ” --Vijay Gill

Why does seamless BGP reconfigurations matter? BGP is critical for ISPs enforce business relationship, responsible for most of traffic BGP configuration is often changed On average, 400+ changes accounted per month in a Tier1 Changing a BGP configuration can impact availability even if the initial and final configurations are safe

Improving network agility with seamless BGP reconfigurations BGP reconfiguration 1 A crash course Finding an ordering 2 Is it easy? Does it exist? Reconfiguration framework 3 Overcome complexity

Improving network agility with seamless BGP reconfigurations BGP reconfiguration 1 A crash course Finding an ordering Is it easy? Does it exist? Reconfiguration framework Overcome complexity

BGP is the only inter-domain routing protocol used today AS10 AS50 Autonomous System AS20 Border Gateway Protocol AS1 AS40 AS30

BGP comes in two flavors AS10 AS50 AS20 AS1 AS40 AS30

external BGP (eBGP) exchanges reachability information between ASes AS10 AS50 AS20 eBGP sessions AS1 AS40 AS30

internal BGP (iBGP) distributes externally learned routes within the AS AS10 AS50 AS20 iBGP sessions AS1 AS40 AS30

Plain iBGP mandates a full-mesh of iBGP sessions O(n 2 ) iBGP sessions where n is the number of routers ... quickly becomes totally unmanageable Fair warning: some sessions are missing

With Route Reflection, iBGP routers are hierarchically organized

Route Reflectors relay route updates between iBGP neighbors Route Reflectors Clients

Route Reflectors relay route updates between iBGP neighbors Route Reflectors Clients Lower layers rely on upper layers to learn and propagate routing informations

iBGP and eBGP need to be carefully configured A BGP configuration is composed of iBGP Clients sessions Route-reflector sessions Peer sessions eBGP External sessions Routing policies

Each part of a BGP configuration can be changed Typical reconfiguration scenarios consist in iBGP Clients sessions Add sessions Route-reflector sessions Remove sessions Peer sessions Change type eBGP External sessions Routing policies

Each part of a BGP configuration can be changed Typical reconfiguration scenarios consist in iBGP Clients sessions Add sessions Route-reflector sessions Remove sessions Peer sessions Change type Add sessions eBGP External sessions Remove sessions Routing policies Modify policies

Reconfiguring BGP can be disruptive BGP reconfigurations can create signaling anomalies [Griffin, SIGCOMM02] dissemination anomalies [Vissicchio, INFOCOM12] forwarding anomalies [Griffin, SIGCOMM02] or any combination of those

Reconfiguring BGP can be disruptive BGP reconfigurations can create signaling anomalies routing oscillations dissemination anomalies black holes forwarding anomalies forwarding loops traffic shifts or any combination of those

Reconfiguring BGP can be disruptive BGP reconfigurations can create signaling anomalies dissemination anomalies How much ? forwarding anomalies or any combination of those

Let’s migrate from a full-mesh to a RR topology

Let’s migrate from a full-mesh to a RR topology, following best practices Establish the RR sessions in a bottom-up manner, [Herrero10] then remove the full-mesh sessions

Best practices do not work Tier1 (50) experiments 1.0 100 (cumul. frequency) 0.8 0.6 60 Loops 60% of the experiments 0.4 were subject to loops for > 35% of the steps 0.2 0 0.0 0 20 40 60 80 100 0 35 100 % of migration steps with anomalies

Best practices do not work Tier1 (50) experiments 1.0 100 (cumul. frequency) 0.8 Traffic shifts 0.6 Loops 100% of the experiments 0.4 were subject to traffic shifts for > 40% of the steps 0.2 0 0.0 0 20 40 60 80 100 0 45 100 % of migration steps with anomalies

Let’s tune BGP policies AS2 P P P E2 E1 E3 P E5 E4 AS1 P AS3 AS4

AS1 learns a destination P via 5 egress points AS2 P P P E2 E1 E3 P E5 E4 AS1 P AS3 AS4

Initially, each egress point is equally preferred AS2 60 60 60 E2 E1 E3 E5 E4 AS1 60 60 AS3 AS4 preference

Depending on its position, each egress receives a percentage of the traffic AS2 60 60 60 20% 10% 40% E2 E1 E3 10% E5 E4 AS1 60 60 10% AS3 AS4 preference usage

Let’s say that AS2 becomes more preferred AS2 60 60 60 20% 10% 40% E2 E1 E3 10% E5 E4 AS1 60 60 10% AS3 AS4 preference usage

Let’s say that AS2 becomes more preferred AS2 60 120 60 20% 10% 40% E2 E1 E3 10% E5 E4 AS1 60 60 10% AS3 AS4 preference usage

Let’s say that AS2 becomes more preferred AS2 60 120 60 0% 0% 100% E2 E1 E3 60% of the traffic -20% -10% experience a traffic shift -10% -10% 0% E5 E4 AS1 60 60 0% AS3 AS4 preference usage

Let’s say that AS2 becomes more preferred AS2 60 120 60 0% 0% 100% E2 E1 E3 0% E5 E4 AS1 60 60 0% AS3 AS4 preference usage

Let’s say that AS2 becomes more preferred AS2 60 120 120 0% 0% 100% E2 E1 E3 0% E5 E4 AS1 60 60 0% AS3 AS4 preference usage

Let’s say that AS2 becomes more preferred AS2 60 120 120 33% 0% 67% E2 60% of the traffic E1 E3 experience a traffic shift -33% 33% of the traffic experience a traffic shift 0% E5 E4 AS1 60 60 0% AS3 AS4 preference usage

Let’s say that AS2 becomes more preferred AS2 120 120 120 33% 0% 67% E2 E1 E3 0% E5 E4 AS1 60 60 0% AS3 AS4 preference usage

Let’s say that AS2 becomes more preferred AS2 120 120 120 28% 16% 56% E2 60% of the traffic -5% E1 E3 experience a traffic shift -11% 33% of the traffic experience a traffic shift 0% E5 E4 16% of the traffic AS1 experience a traffic shift 60 60 0% AS3 AS4 preference usage

During the migration, 109% of the traffic has been shifted AS2 120 120 120 28% 16% 56% E2 60% of the traffic E1 E3 experience a traffic shift 33% of the traffic experience a traffic shift 0% E5 E4 16% of the traffic AS1 experience a traffic shift 60 60 0% AS3 AS4 preference usage

Tuning eBGP policies can create huge traffic shifts 1.0 100 Tier1 experiments (cumul. frequency) 0.8 max LP 0.6 50% of the routers experience > 1 TS 50 for each prefix 0.4 0.2 0.0 0 0.0 0.5 1.0 1.5 2.0 2.5 3.0 0 1 3.0 avg # traffic shifts per router per prefix

Improving network agility with seamless BGP reconfigurations BGP reconfiguration A crash course Finding an ordering 2 Is it easy? Does it exist? Reconfiguration framework Overcome complexity

To avoid reconfiguration problems, a proper operational ordering must be enforced Given an initial & final, anomaly-free, BGP configuration. Find a sequence of configuration changes such that signaling anomalies dissemination anomalies forwarding anomalies never occur, during any migration step

Find a sequence of configuration changes

Find a sequence of configuration changes Does it always exist ?

Find a sequence of configuration changes Does it always exist ? Is it easy to compute ?

We model iBGP configurations by using extended Stable Path Problem instances E2 E2 R1 R2 E1 E1 E1 E2 P P

We model iBGP configurations by using extended Stable Path Problem instances E2 E2 R1 R2 E1 E1 Egress-point E1 E2 to prefix P P P

We model iBGP configurations by using extended Stable Path Problem instances Egress-points E2 E2 in decreasing R1 R2 E1 E1 preference order Egress-point E1 E2 to prefix P P P

We model iBGP configurations by using extended Stable Path Problem instances Egress-points Best- learned E2 E2 in decreasing R1 R2 egress point E1 E1 preference order Egress-point E1 E2 to prefix P P P

A stable BGP configuration determines the forwarding paths being used resulting forwarding paths E2 E2 1 R1 R2 R1 R2 E1 E1 1 2 1 1 E1 E2 E1 E2 P P BGP configuration IGP configuration

A seamless migration ordering might not always exist P P S S RR1 RR2 RR1 RR2 R1 R2 R1 R2 E1 E2 E1 E2 P P P P Initial BGP configuration Final BGP configuration

A seamless migration ordering might not always exist P P S S RR1 RR2 RR1 RR2 R1 R2 R1 R2 removed session E1 E2 E1 E2 P P P P Initial BGP configuration Final BGP configuration

A seamless migration ordering might not always exist P P S S RR1 RR2 RR1 RR2 R1 R2 R1 R2 added session E1 E2 E1 E2 P P P P Initial BGP configuration Final BGP configuration