What TCP/IP Protocol Headers What TCP/IP Protocol Headers What - PowerPoint PPT Presentation

University of North Carolina at University of North Carolina at University of North Carolina at Motivation Motivation Chapel Hill Chapel Hill Chapel Hill Traffic Modeling and Characterization Traffic Modeling and Characterization • Can we continuously acquire network traffic data Can we continuously acquire network traffic data • Can we continuously acquire network traffic data • What TCP/IP Protocol Headers What TCP/IP Protocol Headers What TCP/IP Protocol Headers using off-the-shelf hardware and software? using off-the-shelf hardware and software? using off-the-shelf hardware and software? Can Tell Us About the Web Can Tell Us About the Web Can Tell Us About the Web • • Can we use this information to construct up-to-date, Can we use this information to construct up-to-date, • Can we use this information to construct up-to-date, application-level traffic models? application-level traffic models? application-level traffic models? Félix Hernández Campos – – Populate traffic generator with analytic distributions for Populate traffic generator with analytic distributions for – Populate traffic generator with analytic distributions for F. Donelson Smith simulations and lab experiments simulations and lab experiments simulations and lab experiments Kevin Jeffay • • Can we study the traffic generated by a Can we study the traffic generated by a large large • Can we study the traffic generated by a large David Ott population population of users while protecting their privacy? of users while protecting their privacy? population of users while protecting their privacy? • • Case study: Case study: Web Traffic Web Traffic SIGMETRICS, June 2001 • Case study: Web Traffic SIGMETRICS, June 2001 http://www.cs.unc.edu/Research/dirt http://www.cs.unc.edu/Research/dirt 1 2 1 2 Internet Traffic Characterization Methodology Internet Traffic Characterization Methodology Previous Work Previous Work Trace Acquisition Trace Acquisition • Traffic modeling before the WWW explosion Traffic modeling before the WWW explosion • Study Internet traffic generated by a large and Study Internet traffic generated by a large and • Traffic modeling before the WWW explosion • • Study Internet traffic generated by a large and • diverse population diverse population diverse population – Danzig – Danzig et al. (91, 92) et al. (91, 92) – Danzig et al. (91, 92) – Paxson – Paxson (94) (94) – Paxson (94) • Browsing-based web traffic models • Browsing-based web traffic models • Browsing-based web traffic models – Mah Mah (95) (95) – Mah (95) – – Crovella Crovella et al. (95, 98) et al. (95, 98) University of – Crovella et al. (95, 98) – University of Internet North Carolina North Carolina Internet Gigabit Ethernet Gigabit Ethernet • Models of TCP connections in the web Models of TCP connections in the web • Models of TCP connections in the web • at Chapel Hill at Chapel Hill – Cleveland et al. (00) – Cleveland et al. (00) – Cleveland et al. (00) 35,000 people 35,000 people • Other large-scale trace analyses related to the web • Other large-scale trace analyses related to the web • Other large-scale trace analyses related to the web – Gribble – Gribble & Brewer (97), & Brewer (97), Balakrishnan Balakrishnan et al. (98), et al. (98), Wolman Wolman – Gribble & Brewer (97), Balakrishnan et al. (98), Wolman Traffic Monitor Traffic Monitor et al. (99), and Feldmann Feldmann (00) (00) et al. (99), and Feldmann (00) et al. (99), and ( tcpdump ) ( tcpdump ) 3 3 4 4

Methodology Trace Collection Methodology Trace Collection Benefits of TCP/IP Header Tracing Summary Benefits of TCP/IP Header Tracing Summary • Three sets of traces from Three sets of traces from UNC UNC • Light-weight • Light-weight • Three sets of traces from UNC • • Light-weight – October 99, October 00, April 01 October 99, October 00, April 01 – Off-the-shelf hardware – Off-the-shelf hardware – October 99, October 00, April 01 – – Off-the-shelf hardware – 1 hour-long tracing periods (1-6 GB per trace) 1 hour-long tracing periods (1-6 GB per trace) – Freely available software – Freely available software – 1 hour-long tracing periods (1-6 GB per trace) – – Freely available software – – 42 traces in each set 42 traces in each set – 42 traces in each set • Privacy • Privacy • Privacy – Easy to address by – Easy to address by anonymizing anonymizing IP address offline IP address offline • • Two sets of traces from Two sets of traces from NLANR NLANR (for comparison) (for comparison) – Easy to address by anonymizing IP address offline • Two sets of traces from NLANR (for comparison) – October 99, October 00 – October 99, October 00 – October 99, October 00 • Efficient • Efficient • Efficient – 2 sites 2 sites – 2 sites – – Reduces storage requirements – Reduces storage requirements – Reduces storage requirements » San Diego Supercomputing Center San Diego Supercomputing Center » San Diego Supercomputing Center » » » E.g. 161 GB for headers instead of 803 GB for entire packets E.g. 161 GB for headers instead of 803 GB for entire packets » E.g. 161 GB for headers instead of 803 GB for entire packets » Univ Univ. of Michigan/Merit . of Michigan/Merit » Univ. of Michigan/Merit » – Reduces processing requirements during tracing Reduces processing requirements during tracing – Reduces processing requirements during tracing – – 90 second tracing periods (3-67 MB per trace) 90 second tracing periods (3-67 MB per trace) – 90 second tracing periods (3-67 MB per trace) – » Header extraction and recording only Header extraction and recording only » Header extraction and recording only » – 58 traces in each set – 58 traces in each set – 58 traces in each set • Large-scale Large-scale • Large-scale • – E.g. E.g. 7 days x 12 hr, 1 7 days x 12 hr, 1 Gbps Gbps link (20% link (20% avg avg. . util util.), 35K users .), 35K users – E.g. 7 days x 12 hr, 1 Gbps link (20% avg. util.), 35K users – 5 6 5 6 Trace Collection Case Study: Web Traffic Trace Collection Case Study: Web Traffic Summary Summary Packet Capturing Packet Capturing 99 99 00 00 01 01 • We study a large collection of users as web content We study a large collection of users as web content • We study a large collection of users as web content • consumers Packets Packets consumers consumers Total Total 525 M 1873 M 2419 M 2419 M 525 M 1873 M TCP 85% 91% 91% TCP 85% 91% 91% • We only capture TCP/IP headers • We only capture TCP/IP headers • We only capture TCP/IP headers – No HTTP headers No HTTP headers – No HTTP headers – HTTP HTTP 38% 38% 29% 29% 28% 28% Bytes Bytes Total Total 212 GB 721 GB 721 GB 905 GB 905 GB 212 GB HTTP Request HTTP Request TCP 86% 90% 91% TCP 86% 90% 91% University of University of North Carolina North Carolina HTTP HTTP 56% 56% 35% 35% 36% 36% Internet Internet at Chapel Hill at Chapel Hill Total Traces Size Total Traces Size 36 GB 127 GB 164 GB 164 GB 36 GB 127 GB Web Clients Web Clients Web Servers Web Servers HTTP Response HTTP Response Avg. % of Packets Avg. % of Packets 0 % 0 % 0.02 % 0.003 % 0.02 % 0.003 % Lost by Monitor Lost by Monitor 7 7 8 8