end to end methods for
play

End-to-end Methods for Traffic Shaping Detection, Performance - PowerPoint PPT Presentation

Jul 04, 2023 •171 likes •561 views

End-to-end Methods for Traffic Shaping Detection, Performance Problem Diagnosis, Home Wireless Troubleshooting Partha Kanuparthy Joint work with Constantine Dovrolis AIMS 2011, CAIDA Friday, February 11, 2011 1 Three Tools ShaperProbe :

  1. End-to-end Methods for Traffic Shaping Detection, Performance Problem Diagnosis, Home Wireless Troubleshooting Partha Kanuparthy Joint work with Constantine Dovrolis AIMS 2011, CAIDA Friday, February 11, 2011 1

  2. Three Tools  ShaperProbe : End-to-end detection of traffic shaping GATech, M-Lab (under submission)   Pythia: Detection, localization, diagnosis of performance problems GATech, DoE (early work; 4 months)   Troubleshooting home wireless networks GATech, Intel Labs, CMU (early work; 6 months)  2 Friday, February 11, 2011 2

  3. ShaperProbe: End-to-End Detection of Traffic Shaping Friday, February 11, 2011 3

  4. In this part..  Detecting traffic shapers using active probing (ShaperProbe tool)  ISP case studies 4 Friday, February 11, 2011 4

  5. What is Traffic Shaping?  Practice of dropping link speeds after a burst period  smoothes traffic  helps in managing/reducing congestion  pricing service tiers using shared infrastructure  Why detect shaping?  SLA verification (customers)  configuration testing (operators) Upload: 7Mbps -> 2Mbps in 8s 5 Friday, February 11, 2011 5

  6. Traffic Shapers  Implemented using a Token Bucket at a link  accumulates tokens (bytes) at certain rate (bytes/s)  services packet when it has sufficient tokens Tokens Token  Cisco devices: rate-limit command Bucket  Shapers vs. Policers: Packets  shapers queue packets waiting for tokens; policers drop  we detect both Configuration: burst size, shaping rate 6 Friday, February 11, 2011 6

  7. ShaperProbe: Design Sender (S) sends a constant-rate stream at rate C to receiver (R)  R estimates received rate in small intervals  Probing stops when either:  R sees a level shift in timeseries, or  after 60s  Internet send rate = S R path capacity (C) 7 Friday, February 11, 2011 7

  8. Design: Capacity  Probing rate = path capacity C  We estimate path capacity C before probing:  S sends packet trains of N back-to-back packets  R estimates capacity by measuring dispersion of δ each train: C = ( N − 1) S packet size (1470B) ˆ δ Internet S R δ 8 Friday, February 11, 2011 8

  9. Design: Classification  The probing stream can be designed to emulate well-known applications:  change payload, etc.  e.g., Skype, BitTorrent, ...  some applications may be more likely to be shaped by ISP 9 Friday, February 11, 2011 9

  10. Detecting Shaping  Shaping is characterized by a level shift in received rate  we observe rate in intervals of 300ms  Level shift point if: Received rate ∆  all points before > all points after τ β  min. # points before and after Time Figure 1: Active probing: Level shift detection.  “large” drop in median rate (factor of 1.1): ˜ > γ ˜ R r ( i ) R r ( j ) i =1 ... τ j = τ ...n 10 Friday, February 11, 2011 10

  11. Shaping Configuration  We estimate shaping parameters in case of shaping:  shaping rate: median rate after level shift  burst size: based on bytes sent before level shift τ ρ ] ∆ ± [ R ( i ) − ˆ ρ ] ∆ � σ = ˆ [ R ( i ) − ˆ 2 burst size i =1 (bytes) Received rate ∆ τ shaping rate β (bps) Time Figure 1: Active probing: Level shift detection. 11 Friday, February 11, 2011 11

  12. The ShaperProbe Service  We run a service on M-Lab using 48 server replicas and a load balancer front end  servers connected to tier-1 ASes  Open source client: supported on 3 platforms  Currently 1500+ users a day 12 Friday, February 11, 2011 12

  13. ShaperProbe users say... 13 Friday, February 11, 2011 13

  14. ShaperProbe users say... 13 Friday, February 11, 2011 13

  15. Shaping in ISPs: some observations 14 Friday, February 11, 2011 14

  16. Overview of Data (till Sept. 2010)  M-Lab service has been up for a year (100k+ runs)  We look at a subset of 37,540 runs from 2,000+ ASes  Shaping detections in top-5 ASes in terms of runs: ISP Upstream (%) Dwnstrm. (%) 75.4 82.5 Comcast (3851/5105) (3506/4248) Road Runner 6.4 (69/1073) 63.3 (513/811) AT&T 13.4 (114/849) 17.7 (125/707) Cox 63.4 (399/629) 56.5 (252/446) MCI-Verizon 5.1 (25/490) 7.3 (31/426) 15 Friday, February 11, 2011 15

  17. Shaping factors  There isn’t a “yes-no” answer to “Is my ISP shaping traffic?”  Factors that affect shaping detections in an ISP:  tier of service  geographical region  link type: DSL? cable? Ethernet?  time-of-day  load conditions 16 Friday, February 11, 2011 16

  18. Case study: Comcast 16000 Capacity Capacity 60000 Shaping rate Shaping rate 14000 50000 12000 Rate (Kbps) Rate (Kbps) 40000 10000 8000 30000 6000 20000 4000 10000 2000 16000 0 30000 0 Burst size (KB) Burst size (KB) 14000 25000 12000 20000 10000 8000 15000 6000 10000 4000 5000 2000 0 0 0 500 1000 1500 2000 2500 3000 3500 0 500 1000 1500 2000 2500 3000 3500 Run ID Run ID (a) Upstream. (b) Downstream. C (Mbps) ρ (Mbps) σ (MB) Burst duration (s) C (Mbps) ρ (Mbps) σ (MB) Burst duration (s) 3.5 1 5 16.7 19.4 6.4 10 6.4 4.8 2 5, 10 15.2, 30.5 21.1 12.8 10 10.1 8.8 5.5 10 25.8 28.2 17 20 14.9 14.5 10 10 18.8 34.4 23.4 20 15.3 (a) Upstream. (b) Downstream. ] Comcast High Speed Internet FAQ: PowerBoost . [5] Comcast High-Speed Internet (residential; May 12 ] Comcast Business Class Internet (May 12, 2010) . 2010) . http://customer.comcast.com/Pages/ http://business.comcast.com/internet/ http://www.comcast.com/Corporate/Learn/ FAQListViewer.aspx?topic=Internet&folder= details.aspx . HighSpeedInternet/speedcomparison.html . 8b2fc392-4cde-4750-ba34-051cd5feacf0 . 17 Friday, February 11, 2011 17

  19. Case study: Comcast 16000 Capacity Capacity 60000 Shaping rate Shaping rate 14000 50000 12000 Rate (Kbps) Rate (Kbps) 40000 10000 8000 30000 6000 20000 4000 10000 2000 16000 0 30000 0 Burst size (KB) Burst size (KB) 14000 25000 12000 20000 10000 8000 15000 6000 10000 4000 5000 2000 0 0 0 500 1000 1500 2000 2500 3000 3500 0 500 1000 1500 2000 2500 3000 3500 Run ID Run ID (a) Upstream. (b) Downstream. C (Mbps) ρ (Mbps) σ (MB) Burst duration (s) C (Mbps) ρ (Mbps) σ (MB) Burst duration (s) 3.5 1 5 16.7 19.4 6.4 10 6.4 4.8 2 5, 10 15.2, 30.5 21.1 12.8 10 10.1 8.8 5.5 10 25.8 28.2 17 20 14.9 14.5 10 10 18.8 34.4 23.4 20 15.3 (a) Upstream. (b) Downstream. ] Comcast High Speed Internet FAQ: PowerBoost . [5] Comcast High-Speed Internet (residential; May 12 ] Comcast Business Class Internet (May 12, 2010) . 2010) . http://customer.comcast.com/Pages/ http://business.comcast.com/internet/ http://www.comcast.com/Corporate/Learn/ FAQListViewer.aspx?topic=Internet&folder= details.aspx . HighSpeedInternet/speedcomparison.html . 8b2fc392-4cde-4750-ba34-051cd5feacf0 . 17 Friday, February 11, 2011 17

  20. Case study: Comcast 16000 Capacity Capacity 60000 Shaping rate Shaping rate 14000 50000 12000 Rate (Kbps) Rate (Kbps) 40000 10000 8000 30000 6000 20000 4000 10000 2000 16000 0 30000 0 Burst size (KB) Burst size (KB) 14000 25000 12000 20000 10000 8000 15000 6000 10000 4000 5000 2000 0 0 0 500 1000 1500 2000 2500 3000 3500 0 500 1000 1500 2000 2500 3000 3500 Run ID Run ID (a) Upstream. (b) Downstream. C (Mbps) ρ (Mbps) σ (MB) Burst duration (s) C (Mbps) ρ (Mbps) σ (MB) Burst duration (s) 3.5 1 5 16.7 19.4 6.4 10 6.4 4.8 2 5, 10 15.2, 30.5 21.1 12.8 10 10.1 8.8 5.5 10 25.8 28.2 17 20 14.9 14.5 10 10 18.8 34.4 23.4 20 15.3 (a) Upstream. (b) Downstream. ] Comcast High Speed Internet FAQ: PowerBoost . [5] Comcast High-Speed Internet (residential; May 12 ] Comcast Business Class Internet (May 12, 2010) . 2010) . http://customer.comcast.com/Pages/ http://business.comcast.com/internet/ http://www.comcast.com/Corporate/Learn/ FAQListViewer.aspx?topic=Internet&folder= details.aspx . HighSpeedInternet/speedcomparison.html . 8b2fc392-4cde-4750-ba34-051cd5feacf0 . 17 Friday, February 11, 2011 17

  21. Case study: Comcast 16000 Capacity Capacity 60000 Shaping rate Shaping rate 14000 50000 12000 Rate (Kbps) Rate (Kbps) 40000 10000 8000 30000 6000 20000 4000 10000 2000 16000 0 30000 0 Burst size (KB) Burst size (KB) 14000 25000 12000 20000 10000 8000 15000 6000 10000 4000 5000 2000 0 0 0 500 1000 1500 2000 2500 3000 3500 0 500 1000 1500 2000 2500 3000 3500 Run ID Run ID (a) Upstream. (b) Downstream. C (Mbps) ρ (Mbps) σ (MB) Burst duration (s) C (Mbps) ρ (Mbps) σ (MB) Burst duration (s) 3.5 1 5 16.7 19.4 6.4 10 6.4 4.8 2 5, 10 15.2, 30.5 21.1 12.8 10 10.1 8.8 5.5 10 25.8 28.2 17 20 14.9 14.5 10 10 18.8 34.4 23.4 20 15.3 (a) Upstream. (b) Downstream. ] Comcast High Speed Internet FAQ: PowerBoost . [5] Comcast High-Speed Internet (residential; May 12 ] Comcast Business Class Internet (May 12, 2010) . 2010) . http://customer.comcast.com/Pages/ http://business.comcast.com/internet/ http://www.comcast.com/Corporate/Learn/ FAQListViewer.aspx?topic=Internet&folder= details.aspx . HighSpeedInternet/speedcomparison.html . 8b2fc392-4cde-4750-ba34-051cd5feacf0 . 17 Friday, February 11, 2011 17

Recommend

Generic Methods 36 What are Generic Methods? Generic methods = methods that introduce type

Generic Methods 36 What are Generic Methods? Generic methods = methods that introduce type

Generic Methods 36 What are Generic Methods? Generic methods = methods that introduce type parameters Similar to declaring a generic type but type parameter's scope is limited to method where it is declared The following are

584 views • 6 slides
Clustering ! Hierarchical methods ! Model-based methods ! Density-based methods 1 2 What is

Clustering ! Hierarchical methods ! Model-based methods ! Density-based methods 1 2 What is

Preview ! Introduction Lecture 10 ! Partitioning methods Clustering ! Hierarchical methods ! Model-based methods ! Density-based methods 1 2 What is Clustering? Examples of Clustering Applications ! Cluster: a collection of data objects !

301 views • 8 slides
DESIGN & ANALYSIS METHODS DESIGN & ANALYSIS METHODS FOR DESIGN & ANALYSIS METHODS

DESIGN & ANALYSIS METHODS DESIGN & ANALYSIS METHODS FOR DESIGN & ANALYSIS METHODS

SNAME Philadelphia Section Meeting May 2013 y USA (originally presented at MTU Naval Symposium, September 2011, GERMANY) DESIGN & ANALYSIS METHODS DESIGN & ANALYSIS METHODS FOR DESIGN & ANALYSIS METHODS DESIGN & ANALYSIS

1.11k views • 80 slides
Why mixed methods? Methods should be mixed in a way that has complementary strengths and

Why mixed methods? Methods should be mixed in a way that has complementary strengths and

Mixed, mixing and mul.plying methods Dr Gemma Derrick Centre for Higher Educa.on Research & Evalua.on Why mixed methods? Methods should be mixed in a way that has complementary strengths and non-overlapping weaknesses. It involves the

239 views • 11 slides
Meshless Meshless Methods Meshless Meshless Methods Methods Methods Contents

Meshless Meshless Methods Meshless Meshless Methods Methods Methods Contents

Meshless Meshless Methods Meshless Meshless Methods Methods Methods Contents Introduction Mess Free Methods Element Free Galerkin Method Element Free Galerkin Method Moving Particle Semi-Implicit Method Conclusion C l i

562 views • 38 slides
Monte Carlo Methods An introduction to Monte Carlo (MC) methods How to use MC methods

Monte Carlo Methods An introduction to Monte Carlo (MC) methods How to use MC methods

Numerical and Scientific Computing with Applications David F . Gleich CS 314, Purdue September 12, 2016 In this class: Monte Carlo Methods An introduction to Monte Carlo (MC) methods How to use MC methods Next class to estimate

826 views • 17 slides
Direct Search Methods (nongradient methods) 1. Random search methods 2. Univariate method (one

Direct Search Methods (nongradient methods) 1. Random search methods 2. Univariate method (one

Unconstrained Minimization in R n Direct Search Methods (nongradient methods) 1. Random search methods 2. Univariate method (one variable at a time) 3. Pattern search methods a) Hooke and Jeeves method b) Powells conjugate direction method

154 views • 12 slides
Methods Constructors Instance methods Class methods 30 Constructors have the same

Methods Constructors Instance methods Class methods 30 Constructors have the same

Methods Constructors Instance methods Class methods 30 Constructors have the same name as the class do not return anything they begin with a call to the constructor from the parent class (super(...)) public class MyClass{

563 views • 24 slides
XMLTree Methods 7 January 2019 OSU CSE 1 Methods for XMLTree All the methods for XMLTree are

XMLTree Methods 7 January 2019 OSU CSE 1 Methods for XMLTree All the methods for XMLTree are

XMLTree Methods 7 January 2019 OSU CSE 1 Methods for XMLTree All the methods for XMLTree are instance methods , i.e., you call them as follows: t.methodName(arguments) where t is an initialized variable of type XMLTree 7 January 2019 OSU

1.14k views • 65 slides
CSE CSE 460 460 Evolutionary Evolutionary Methods Methods In this section we will look at

CSE CSE 460 460 Evolutionary Evolutionary Methods Methods In this section we will look at

CSE CSE 460 460 Evolutionary Evolutionary Methods Methods In this section we will look at evolutionary methods Genetic Algorithms Evolution Strategies Genetic Programming Constraints in Evolutionary Methods References

871 views • 57 slides
Benefits of Methods Chapter 5 Methods Write a method once and reuse it anywhere.

Benefits of Methods Chapter 5 Methods Write a method once and reuse it anywhere.

Benefits of Methods Chapter 5 Methods Write a method once and reuse it anywhere. Information hiding: hide the implementation from the user Reduce complexity 1 4 Motivating Example Defining and Using Methods Often we need to

232 views • 12 slides
Roadmap SE process management Waterfall model Incremental methods Agile/XP methods,

Roadmap SE process management Waterfall model Incremental methods Agile/XP methods,

Roadmap SE process management Waterfall model Incremental methods Agile/XP methods, SCRUM Iterative / spiral methods (eg, RUP) Evolutionary methods V -Model CMMI 320312 Software Engineering (P. Baumann) 1 The

339 views • 19 slides
Stochastic Simulation Methods: Variance reduction methods Antithetic variables Bo Friis

Stochastic Simulation Methods: Variance reduction methods Antithetic variables Bo Friis

Variance reduction methods Variance reduction methods To obtain better estimates with the same ressources Exploit analytical knowledge and/or correlation Stochastic Simulation Methods: Variance reduction methods Antithetic

276 views • 4 slides
Methods Mysteries Revealed Wri2ng monolithic programs in the

Methods Mysteries Revealed Wri2ng monolithic programs in the

Methods A method (a.k.a. func2on, procedure, subrou2ne) is a piece of code that performs Methods and Data a useful ac2on. (Savitch, Chapter 5) Up to this

144 views • 10 slides
Carlo (MC) methods Introduction to MC methods Why Scientists like to gamble Monte Carlo Methods

Carlo (MC) methods Introduction to MC methods Why Scientists like to gamble Monte Carlo Methods

Introduction to Monte Carlo (MC) methods Introduction to MC methods Why Scientists like to gamble Monte Carlo Methods 2 Overview Integration by random numbers Why? How? Uncertainty, Sharply peaked distributions Importance

1.22k views • 36 slides
Carlo (MC) methods Introduction to MC methods Why Scientists like to gamble Monte Carlo Methods

Carlo (MC) methods Introduction to MC methods Why Scientists like to gamble Monte Carlo Methods

Introduction to Monte Carlo (MC) methods Introduction to MC methods Why Scientists like to gamble Monte Carlo Methods 2 Overview Integration by random numbers Why? How? Uncertainty, Sharply peaked distributions Importance

785 views • 37 slides
Chapter 5: Monte Carlo Methods Monte Carlo methods are learning methods Experience

Chapter 5: Monte Carlo Methods Monte Carlo methods are learning methods Experience

Chapter 5: Monte Carlo Methods Monte Carlo methods are learning methods Experience values, policy Monte Carlo methods can be used in two ways: ! model-free: No model necessary and still attains optimality ! Simulated: Needs only a

2.17k views • 32 slides
R Regression Methods Interrogate R Output Objects Paul E. Johnson Center for Research Methods

R Regression Methods Interrogate R Output Objects Paul E. Johnson Center for Research Methods

Regression Methods 1 / 72 R Regression Methods Interrogate R Output Objects Paul E. Johnson Center for Research Methods and Data Analysis University of Kansas 2012 Regression Methods 2 / 72 Outline 1 Methods 2 Interrogate Models

844 views • 72 slides
Research Methods E VAN S TRASNICK CS 347 * Your paper is an argument Your methods provide

Research Methods E VAN S TRASNICK CS 347 * Your paper is an argument Your methods provide

Research Methods E VAN S TRASNICK CS 347 * Your paper is an argument Your methods provide evidence Different arguments require different evidence ...and more! From McGrath, Methodology Matters Method triangulation All methods are flawed

576 views • 32 slides
7. Iterative Methods: Roots and Optima Citius, Altius, Fortius! 7. Iterative Methods: Roots and

7. Iterative Methods: Roots and Optima Citius, Altius, Fortius! 7. Iterative Methods: Roots and

Relaxation Methods Minimization Methods Non-Linear Equations Multigrid Applications 7. Iterative Methods: Roots and Optima Citius, Altius, Fortius! 7. Iterative Methods: Roots and Optima Numerical Programming I (for CSE), Hans-Joachim

846 views • 42 slides
Methods for Anomaly Detection: a Survey Kalinichenko L.A., Shanin I., Taraban I. IPI RAN

Methods for Anomaly Detection: a Survey Kalinichenko L.A., Shanin I., Taraban I. IPI RAN

RCDL 2014 Methods for Anomaly Detection: a Survey Kalinichenko L.A., Shanin I., Taraban I. IPI RAN Outline Introduction Motivation Methods taxonomy Metric Data Oriented Methods Distance-based Data Methods

714 views • 34 slides
25. NLP algorithms Overview Local methods Constrained optimization Global methods

25. NLP algorithms Overview Local methods Constrained optimization Global methods

CS/ECE/ISyE 524 Introduction to Optimization Spring 201718 25. NLP algorithms Overview Local methods Constrained optimization Global methods Black-box methods Course wrap-up Laurent Lessard (www.laurentlessard.com)

649 views • 35 slides
MODULE 14: PROCUREMENT METHODS AND IECC OVERVIEW A c c r e d i t e d T e x a s E n e r g y M a

MODULE 14: PROCUREMENT METHODS AND IECC OVERVIEW A c c r e d i t e d T e x a s E n e r g y M a

MODULE 14: PROCUREMENT METHODS AND IECC OVERVIEW A c c r e d i t e d T e x a s E n e r g y M a n a g e r Outline Procurements & Delivery Methods Procurement Methods Professional Services Procurement Methods Performance

240 views • 20 slides
Estimating the Survival Function One-sample nonparametric methods: We will consider three methods

Estimating the Survival Function One-sample nonparametric methods: We will consider three methods

Estimating the Survival Function One-sample nonparametric methods: We will consider three methods for estimating a survivorship function S ( t ) = Pr ( T t ) without resorting to parametric methods: (1) Kaplan-Meier (2) Life-table (Actuarial

1.29k views • 71 slides

More recommend