Archipelago Measurement Infrastructure Updates and Analyses Young Hyun CAIDA ISMA 2009 AIMS Workshop Feb 12, 2009
Outline Focus and Architecture Monitor Deployment Measurements Future Work 2
Introduction Archipelago (Ark) is CAIDA’s next-generation active measurement infrastructure evolution of the skitter infrastructure in production since Sep 12, 2007 3
Focus easy development and rapid prototyping lower barriers => implement better measurements faster with lower cost • measurement infrastructures notoriously lack funding raise level of abstraction with high-level API and scripting language • inspiration from Scriptroute, Metasploit, Scapy, Racket 4
Focus dynamic and coordinated measurements take advantage of multiple distributed measurement nodes in sophisticated ways • one measurement triggers another measurement • use multiple nodes to divide and conquer • synchronize measurements for example: Doubletree; tomography; Rocketfuel-like targeted discovery of a single network’s topology 5
Focus measurement services build upon the work of others; share services between measurement activities • for example, on-demand traceroute/ping service; IP-to-AS mapping service similiar in goal to service-oriented architecture (SOA) but at finer granularity and without the complexity 6
Architecture Ark is composed of measurement nodes (machines) located in various networks worldwide many thanks to the organizations hosting Ark boxes please contact us if you want to host an Ark box Ark employs a tuple space to enable communication and coordination a tuple space is a distributed shared memory combined with a small number of easy-to-use operations a tuple space stores tuples, which are arrays of simple values (strings and numbers), and clients retrieve tuples by pattern matching 7
Architecture use tuple space for decentralized (that is, peer-to- peer) communication, interaction, and coordination monitor 4 monitor 5 central server r 3 o t i n o m m o n i t o r 1 monitor 2 8
Monitor Deployment 33 monitors in 22 countries Continent Organization 12 North America 19 academic 2 South America 9 research network 11 Europe 2 network infrastructure 1 Africa 1 commercial network 5 Asia 1 community network 2 Oceania 1 military research 9
Measurements IPv4 Routed /24 Topology IPv4 Routed /24 AS Links IPv 6 Topology DNS Names DNS Query/Response Traffic Spoofer Project Collaboration 10
IPv4 Routed /24 Topology ongoing large-scale topology measurements ICMP Paris traceroute to every routed /24 (7.4 million) running scamper • written by Matthew Luckie of WAND, University of Waikato group monitors into teams and dynamically divide up the measurement work among team members 13-member team probes every /24 in 48 hrs at 100pps only one monitor probes each /24 per cycle 3 teams active 11
IPv4 Routed /24 Topology nap-it (3) eug-us (3) dfw-us (3) 30 pna-es (3) her-gr (3) scl-cl (3) she-cn (3) yow-ca (2) 25 tpe-tw (2) ams-nl (2) bwi-us (2) zrh-ch (2) sjc-us (2) 20 gig-br (2) cmn-ma (2) hnl-us (2) lax-us (2) cbg-uk (2*) 15 vie-at (2) iad-us (2*) yto-ca (1*) bcn-es (1*) hlz-nz (1) 10 lej-de (1) laf-us (1*) syd-au (1*) san-us (1*) nrt-jp (1*) 5 mnl-ph (1*) hel-fi (1*) dub-ie (1) cjj-kr (1*) amw-us (1) 0 Sep Nov Jan Mar May Jul Sep Nov Jan Mar 07 07 08 08 08 08 08 08 09 09 Sep 2007 to Jan 2009 (17 months): 2.5 billion traceroutes; 1.0TB data 12
IPv4 Routed /24 Topology nap-it (3) eug-us (3) dfw-us (3) software 30 pna-es (3) her-gr (3) failure scl-cl (3) she-cn (3) yow-ca (2) 25 tpe-tw (2) ams-nl (2) bwi-us (2) zrh-ch (2) sjc-us (2) 20 gig-br (2) cmn-ma (2) hnl-us (2) lax-us (2) cbg-uk (2*) 15 vie-at (2) iad-us (2*) yto-ca (1*) bcn-es (1*) hlz-nz (1) 10 lej-de (1) laf-us (1*) syd-au (1*) san-us (1*) nrt-jp (1*) 5 mnl-ph (1*) hel-fi (1*) dub-ie (1) cjj-kr (1*) amw-us (1) 0 Sep Nov Jan Mar May Jul Sep Nov Jan Mar 07 07 08 08 08 08 08 08 09 09 Sep 2007 to Jan 2009 (17 months): 2.5 billion traceroutes; 1.0TB data 13
IPv4 Routed /24 Topology nap-it (3) eug-us (3) dfw-us (3) hardware 30 pna-es (3) her-gr (3) failure scl-cl (3) she-cn (3) yow-ca (2) 25 tpe-tw (2) ams-nl (2) bwi-us (2) zrh-ch (2) sjc-us (2) 20 gig-br (2) cmn-ma (2) hnl-us (2) lax-us (2) cbg-uk (2*) 15 vie-at (2) iad-us (2*) yto-ca (1*) bcn-es (1*) hlz-nz (1) 10 lej-de (1) laf-us (1*) syd-au (1*) san-us (1*) nrt-jp (1*) 5 mnl-ph (1*) hel-fi (1*) dub-ie (1) cjj-kr (1*) amw-us (1) 0 Sep Nov Jan Mar May Jul Sep Nov Jan Mar 07 07 08 08 08 08 08 08 09 09 Sep 2007 to Jan 2009 (17 months): 2.5 billion traceroutes; 1.0TB data 14
IPv4 Routed /24 Topology nap-it (3) replacement eug-us (3) dfw-us (3) 30 pna-es (3) power supply her-gr (3) scl-cl (3) she-cn (3) died yow-ca (2) 25 tpe-tw (2) ams-nl (2) bwi-us (2) power supply zrh-ch (2) sjc-us (2) 20 gig-br (2) died cmn-ma (2) hnl-us (2) lax-us (2) cbg-uk (2*) 15 vie-at (2) iad-us (2*) yto-ca (1*) bcn-es (1*) hlz-nz (1) 10 lej-de (1) laf-us (1*) syd-au (1*) san-us (1*) nrt-jp (1*) 5 mnl-ph (1*) hel-fi (1*) dub-ie (1) cjj-kr (1*) amw-us (1) 0 Sep Nov Jan Mar May Jul Sep Nov Jan Mar 07 07 08 08 08 08 08 08 09 09 Sep 2007 to Jan 2009 (17 months): 2.5 billion traceroutes; 1.0TB data 15
IPv4 Routed /24 AS Links AS links from Routed /24 Topology traces map IP addresses to ASes with RouteViews BGP table 16
IPv4 Routed /24 AS Links statistics for 1 month of AS links from three sources (Dec 2008): average max average mean nodes links neighbor degree degree clustering degree 23,425 56,760 2,509 4.85 467.3 0.354 Ark 22,995 74,140 3,590 6.45 705.4 0.446 DIMES RouteViews 30,760 65,775 2,328 4.28 487.2 0.241 (rv2) “avg neighbor deg” = avg neighbor degree of the avg k - degree node averaged over all k “mean clustering” = (avg number of links between neighbors of k -deg nodes) / (max possible such links for k ) averaged over all k 17
3 AS Links Sources: 1 Month 10 0 DIMES AS links (2008-12) Ark AS links (2008-12) RouteViews (rv2) AS links (2008-12) 10 -1 10 -2 CCDF 10 -3 10 -4 10 -5 10 0 10 1 10 2 10 3 10 4 Node degree 18
3 AS Links Sources: 1 Month 10 3 DIMES AS links (2008-12) Ark AS links (2008-12) RouteViews (rv2) AS links (2008-12) average neighbor degree 10 2 10 1 10 0 10 0 10 1 10 2 10 3 10 4 Node degree 19
3 AS Links Sources: 1 Month 10 0 DIMES AS links (2008-12) Ark AS links (2008-12) RouteViews (rv2) AS links (2008-12) 10 -1 clustering 10 -2 10 -3 10 -4 10 0 10 1 10 2 10 3 10 4 Node degree 20
AS Links Growth AS links seem to accumulate linearly without bound in skitter, Ark, DIMES; possibly in BGP even with fixed traceroute sources and destination list (which happened with skitter for 4 years) AS graph densification: average degree increases for example: 1 year of Ark (2008): 104k AS links, 28k ASes 2 years of DIMES: 356k AS links, 29k ASes 7.5 years of skitter: 209k AS links, 27k ASes 21
AS Links Growth hard to determine the “natural” time period to aggregate AS links 1 month? 6 months? years? when do we get a representative AS graph? 22
AS Links Growth hard to compare different infrastructures you can always make AS graph bigger by aggregating 23
AS Links Growth hard to compare different infrastructures you can always make AS graph bigger by aggregating in fact, got spam on this ... 23
AS Links Growth hard to compare different infrastructures you can always make AS graph bigger by aggregating in fact, got spam on this ... 23
Ark AS Links Growth 29000 105000 # nodes # links 100000 28000 95000 90000 27000 85000 # nodes # links 26000 80000 75000 25000 70000 65000 24000 60000 23000 55000 1 2 3 4 5 6 7 8 9 10 11 12 Months of accumulation 24
Ark AS Links Growth 4500 7.5 max degree average degree 4250 7 4000 3750 6.5 average degree max degree 3500 6 3250 3000 5.5 2750 5 2500 2250 4.5 1 2 3 4 5 6 7 8 9 10 11 12 Months of accumulation 25
Ark AS Links Growth 850 0.54 average neighbor degree clustering 0.52 800 0.5 750 average neighbor degree 0.48 700 0.46 clustering 650 0.44 0.42 600 0.4 550 0.38 500 0.36 450 0.34 1 2 3 4 5 6 7 8 9 10 11 12 Months of accumulation 26
Ark AS Links: 1, 6, 12 Months 10 0 Ark AS links (2008, 1 to 12) Ark AS links (2008, 7 to 12) Ark AS links (2008, 12 to 12) 10 -1 10 -2 CCDF 10 -3 10 -4 10 -5 10 0 10 1 10 2 10 3 10 4 Node degree 27
Ark AS Links: 1, 6, 12 Months 10 3 Ark AS links (2008, 1 to 12) Ark AS links (2008, 7 to 12) Ark AS links (2008, 12 to 12) average neighbor degree 10 2 10 1 10 0 10 1 10 2 10 3 10 4 Node degree 28
Ark AS Links: 1, 6, 12 Months 10 0 Ark AS links (2008, 1 to 12) Ark AS links (2008, 7 to 12) Ark AS links (2008, 12 to 12) 10 -1 clustering 10 -2 10 -3 10 0 10 1 10 2 10 3 10 4 Node degree 29
Ark IPv 6 Topology ongoing “large-scale” IPv6 measurements since Dec 12, 2008 6 monitors: 3 in US, 3 in Europe 2 IPv6 boxes down 3 more IPv6 boxes coming Real Soon Now ICMP Paris traceroute to every routed prefix each monitor probes a random destination in every routed prefix in every cycle; 1,553 prefixes <= /48 reduced probing rate to take 2 days per cycle running scamper 30
Recommend
More recommend
