Quaking Tables: The Taiwan Earthquakes and the Internet Routing - PowerPoint PPT Presentation

Quaking Tables: The Taiwan Earthquakes and the Internet Routing Table Alin Popescu, Renesys Corp APRICOT Bali 2007 Todd Underwood, Renesys Corp Earl Zmijewski, Renesys Corp

Overview • Large earthquakes hit Luzon Strait, south of Taiwan on 26 December 2006 • Seven of nine cables passing through the straight were severed • We review the event from a perspective of the Internet Routing tables • Routing outages occurred, significant congestion was reported, instability persisted • Recovery was delayed and uneven Quakes and Routing Page 2 February, 2007

Submarine cables in East Asia Page 3 • Two of nine cables not impacted:  Asia Netcom's EAC  Guam-Philippines • All cables reported repaired as of February 14, 2007 (source: Office of the Telecommunications Authority of Hong Kong) Quakes and Routing Page 3 February, 2007

Submarine cables in East Asia (2) Quakes and Routing February, 2007

Repairing submarine cables is difficult! Page 5 Image credit: Alcatel Quakes and Routing Page 5 February, 2007

Definitions: Outage, unreachable, unstable • A network outage occurs when routes to the network Page 6 are withdrawn by a large number of BGP routers worldwide. • In this case if no less specific route is available, the network is unreachable and effectively disconnected from all or parts of the Internet. • Unstable networks are not completely disconnected, but show frequent changes in network routing paths or alternating announcements and withdrawals ( route flapping ) – serious packet losses. Quakes and Routing Page 6 February, 2007

Timeline • Six earthquakes of magnitude 5.0 or higher hit the Taiwan region (all times UTC): • 12/26 12:26:21 7.1 -- main quake • 12/26 12:34:14 6.9 • 12/26 12:40:22 5.5 • 12/26 15:41:44 5.4 • 12/26 17:35:10 5.4 • 12/27 02:30:39 5.6 • 12/28 16:51:16 4.4 • Outaged prefixes ramp up from 400 to almost 1200 from the first quake through seventh Quakes and Routing Page 7 February, 2007

Timeline (2) • 03:31 27 Dec 2006: 60 mins after the last quake, outaged network count spikes to 4k • The “spike” is short-lived (< 2 hrs) but > 2k prefixes out for 6 hours. • 31 Dec 2006 12:00: Outages return to pre-quake levels. • Instability level remains high into January. Quakes and Routing Page 8 February, 2007

Data Collection Infrastructure • 165+ peering sessions from 105+ different ASNs • In this talk, we focus on East Asian prefixes only Quakes and Routing Page 9 February, 2007

• Disasters Have Signatures • Sharp onset associated with some real-world event • Slow return to baseline • Varies considerably • Power outages: fast • Major natural disasters, much slower • Noise in the recovery (not in the onset) Quakes and Routing Page 10 February, 2007

• Power (Northeast US, 2003) Quakes and Routing Page 11 February, 2007

• Hurricane (Katrina, 2005) Quakes and Routing Page 12 February, 2007

The Pattern of the Taiwan Quakes • Ramping up outages and spikes in instabilities • Gradual increase in number of outages after major quake in Dec. 26 • Big spike in outages/unstables associated with smaller quake on Dec. 27 • Recovery typically noisy • Pattern was probably affected by the number of different cable systems involved – this is not really one event but at least seven. Quakes and Routing Page 13 February, 2007

• Outages & Quakes – 3 Day Quakes and Routing Page 14 February, 2007

• Outages & Quakes – 10 Day Quakes and Routing Page 15 February, 2007

• Outages by Country – 3 Day Quakes and Routing Page 16 February, 2007

Why India? Major subcontinent bandwidth heads East Image credit: Asia Netcom Quakes and Routing Page 17 February, 2007

• Outages by Country – 10 Day Quakes and Routing Page 18 February, 2007

• Outages by Country – Peak Quakes and Routing Page 19 February, 2007

Outages by Origin ASN – 3 day Quakes and Routing Page 20 February, 2007

Outage by Origin ASN – 10 day Quakes and Routing Page 21 February, 2007

Outages by Origin ASN – Peak Quakes and Routing Page 22 February, 2007

• Instability Metrics • Use algorithm described in http://www.nanog.org/mtg-0402/ogielski.html • PenaltyBox(T,K,H,C): the number of globally routable prefixes at time T that have flap penalty K, using the classic flap dampening algorithm with half-life of H and ceiling of C. • Reasonable: H=600s, C=15, K={0,…,C}. • (Basically flap-dampening algorithm for scoring the “penalty” of a prefix). • Shown in graphs are pfxs w/ inst >= 3 Quakes and Routing Page 23 February, 2007

• Unstables & Quakes – 3 Day Quakes and Routing Page 24 February, 2007

• Unstables & Quakes – 10 Day Quakes and Routing Page 25 February, 2007

• Unstables by Country – 3 Day Quakes and Routing Page 26 February, 2007

• Unstables by Country – 10 Day Quakes and Routing Page 27 February, 2007

• Unstables by Country – Peak Quakes and Routing Page 28 February, 2007

Unstables by Origin ASN – 3 day Quakes and Routing Page 29 February, 2007

Unstables by Origin ASN – 10 day Quakes and Routing Page 30 February, 2007

Unstables by Origin ASN – Peak Quakes and Routing Page 31 February, 2007

• Winners & Losers: By Country • Used maximum to median ratio of outages and unstable networks • Worst Impacted: • China, Hong Kong • Least Impacted: • Korea, Japan, Malaysia Quakes and Routing Page 32 February, 2007

• Winners & Losers: By Country (cont'd) Max/Median Outages by Country Max/Median Unstables by Country 70 1400 65 1300 Max/Median Unstables Max/Median Outages 60 1200 55 1100 50 1000 45 900 40 800 35 700 30 600 25 500 20 400 15 300 200 10 100 5 0 0 CN HK PH SG ID BD IN TH TW VN JP PK MY KR HK TW ID VN CN IN SG PK TH BD PH MY JP KR Country Country Quakes and Routing Page 33 February, 2007

Impacted ASNs • Examined Asian prefixes outaged and/or unstable by origin AS – 1667 ASNs impacted: • China Telecom: AS4134, AS4812 (CN) • Sify: AS9583 (IN) • VSNL: AS4755 (IN) • Bharti BT Internet: AS9498 (IN) • PT Telekomunikasi: AS17974 (ID) • CNC Group (AS4808, AS4837) (CN) • Smart Broadband: AS10139 (PH) • INDOSAT: AS4795 (ID) Quakes and Routing Page 34 February, 2007

• Winners & Losers by ASN (cont'd) 4134 4134 9583 9583 4755 4755 9498 9498 4812 4812 17974 17974 4808 4808 4837 4837 10139 10139 4795 4795 9304 9304 17488 17488 10091 10091 9929 9929 9394 9394 9829 9829 24138 24138 4657 4657 4515 4515 9381 9381 0 500 1000 1500 0 25 50 75 100 % of total prefixes # of prefixes Quakes and Routing Page 35 February, 2007

Edge Analysis • PPT (Prefix, Peer, Time) score for each edge: for each prefix, for each peer, sum the amount of time the peer saw the prefix routed on the edge during a time interval • Caveats: • All prefixes have the same weight • Cannot distinguish between an edge with a lot of prefixes seen by only few peers, and an edge with few prefixes seen by a lot of peers Quakes and Routing Page 36 February, 2007

Edge Analysis (cont'd) • For each edge, generate time series with range 25 Dec 2006 to 15 Jan 2007 and 8 hours resolution • Considered only prefixes from South-East Asia and the Indian subcontinent • Filter out edges seen by less than 20 peers • Evaluate the time series using the 2-day median difference between end and beginning Quakes and Routing Page 37 February, 2007

Top 10 Edge Winners Quakes and Routing Page 38 February, 2007

Top 10 Edge Losers Quakes and Routing Page 39 February, 2007

Winning Edges Top 20 Winners 1239_9498 1273_7473 6762_7473 3320_9498 1273_4651 6762_4651 3561_4755 AS Edges 3356_701 6762_24077 2914_4657 3356_1239 701_4761 7018_4755 4755_9583 15412_9304 4637_4761 3561_1239 15412_18101 701_9498 3561_4134 0 500,000 1,000,000 1,500,000 2,000,000 2,500,000 3,000,000 3,500,000 PPT Score / 100,000 Quakes and Routing Page 40 February, 2007

Losing Edges Top 20 Losers 22822_7473 3356_4837 1239_4657 1239_4637 1239_9929 3257_3561 1299_7018 AS Edges 2914_4755 9930_ORIGIN 3549_7473 7018_4761 7473_4761 7018_9583 3320_9929 3491_4761 1239_4134 6453_4651 3356_7473 3561_7473 7473_9498 -7,000,000 -6,000,000 -5,000,000 -4,000,000 -3,000,000 -2,000,000 -1,000,000 0 PPT Score / 100,000 Quakes and Routing Page 41 February, 2007

Regional Stories • AS7473 Singapore Telecom (SG) • AS4134 China Telecom (CN) • AS9498 Bharti BT Internet (IN) • AS4761 INDOSAT (ID) • AS4651 Communication Authority of Thailand (TH) • AS24077 TMHK Global Transit (HK) Quakes and Routing Page 42 February, 2007

Singapore Telecom (AS7473) Quakes and Routing Page 43 February, 2007

China Telecom (AS4134) Quakes and Routing Page 44 February, 2007

Bharti BT Internet (AS9498) Quakes and Routing Page 45 February, 2007

INDOSAT (AS4761) Quakes and Routing Page 46 February, 2007

Communication Authority of Thailand (AS4651) Quakes and Routing Page 47 February, 2007