On the Predictability of Large Transfer TCP Throughput Qi He Constantine Dovrolis Mostafa Ammar College of Computing Georgia Institute of Technology SIGCOMM '05 1
Outline TCP throughput prediction: problem statement and motivation Formula-Based (FB) prediction A formula-based predictor Types of FB prediction errors Experimental evaluation History-Based (HB) prediction Typical history-based predictors Dealing with outliers and level shifts Experimental evaluation Predictability factors What makes some paths less predictable than others? SIGCOMM '05 2
Problem Statement and Motivation Objective: Predict the throughput of a bulk TCP transfer on a given path Motivation: Server selection Overlay/multi-homed routing Load balancing Grid computing P2P downloading SIGCOMM '05 3
Constraints and Assumptions Prediction is needed before the start of transfer Performing “test TCP transfer” just for prediction is too intrusive/slow Measuring certain “lightweight path characteristics” (e.g., loss rate or RTT) is not intrusive SIGCOMM '05 4
Two Classes of TCP Throughput Predictors Prediction Basis Inputs Advantages Issues Method Analytical Formula Estimates of No previous Prediction model for Based path’s RTT transfers TCP accuracy? and loss rate required (FB) throughput History of Prediction History Time series previous TCP based on Prediction Based forecasting transfers on actual TCP accuracy? theory the same (HB) transfers path SIGCOMM '05 5
Main Contributions Evaluate prediction accuracy for FB and HB predictions FB can be significantly inaccurate, especially for congestion-limited flows HB is quite accurate even with simple linear predictors and sporadic previous samples Explain major causes of prediction errors in terms of underlying network and TCP behavior Focus on cause-effect relations, rather than black box evaluation Study effects of path properties and transfer characteristics on prediction accuracy Load, degree of multiplexing Receiver window, transfer frequency SIGCOMM '05 6
Outline TCP throughput prediction: problem statement and motivation Formula-Based (FB) prediction A formula-based predictor Types of FB prediction errors Experimental evaluation History-Based (HB) prediction Typical history-based predictors Dealing with outliers and level shifts Experimental evaluation Predictability factors What makes some paths less predictable than others? SIGCOMM '05 7
TCP Throughput Model Analytical model of the expected TCP throughput R as a function of several path characteristics R = f (T, p) ( p > 0) T, p : RTT and loss rate experienced during the flow We use PFTK model by Padhye et. al (Sigcomm ’98) M W ( ) = > R min , , p 0 T 2 bp 3 bp + + 2 T T min( 1 , ) p ( 1 32 p ) 0 3 8 M: path MTU (Maximum Transfer Unit) W: TCP maximum congestion window T 0: TCP retransmission timeout b: segments released per new ACK SIGCOMM '05 8
An FB Predictor Measure loss rate p’, RTT T’ before the target flow starts Typical measurement: periodic probing, e.g., Ping Apply T’ and p’ to the throughput equation ˆ = f (T’, p’) ( p ’ > 0 ) R With the PFTK model… M W ^ = > R min( , ), p ' 0 T ' 2 bp ' 3 bp ' + + 2 T ' T min( 1 , ) p ' ( 1 32 p ' ) 0 3 8 SIGCOMM '05 9
An FB Predictor Measure loss rate p’, RTT T’ before the target flow starts Typical measurement: periodic probing, e.g., Ping Apply T’ and p’ to the throughput equation ˆ = f (T’, p’) ( p ’ > 0 ) R With the PFTK model… M W > min( , ), if p ' 0 T ' 2 bp ' 3 bp ' + + 2 T ' T min( 1 , ) p ( 1 32 p ' ) ^ = R 0 3 8 W = min( A ' , ) , if p ' 0 T ' A’: available bandwidth estimation SIGCOMM '05 10
Potential Issues with FB Prediction Differences between T’ and T, p’ and p T’, p’ T, p Temporal: before flow during flow Sampling: periodic probing TCP “sampling” Issue Effect Additional load of the target Overestimate throughput flow may increase T, p Adaptive and bursty TCP Underestimate or sampling vs. non-adaptive overestimate throughput periodic sampling SIGCOMM '05 11
Evaluation of Throughput Prediction Accuracy Epoch: IPerf for TCP transfers, pathload for available bandwidth, ping (interval: 100ms, pkt size: 41bytes) for RTT & loss rate TCP Throughput R, RTT Available bandwidth A' RTT T' and Loss Rate p' T e and Loss Rate p e ( pathload , 20s-60s) ( ping , 60s) ( ping and iperf , 60s ) One Measurement Epoch SIGCOMM '05 12
Evaluation of Throughput Prediction Accuracy Epoch: IPerf for TCP transfers, pathload for available bandwidth, ping (interval: 100ms, pkt size: 41bytes) for RTT & loss rate TCP Throughput R, RTT Available bandwidth A' RTT T' and Loss Rate p' T e and Loss Rate p e ( pathload , 20s-60s) ( ping , 60s) ( ping and iperf , 60s ) 150 epochs Each trace consists of 150 consecutive epochs We used 35 Internet paths; 7 traces on each path; hosts in US, Europe, Korea SIGCOMM '05 13
Prediction Error Metrics Relative Error ^ − R R = E ^ min( R , R ) ^ ^ R R =(1/w) R, and = wR both have: |E|=w-1 ^ ^ R R e.g., = ½ R, and = 2 R both have: |E|=1 Root Mean Square Relative Error n 1 ∑ 2 RMSRE = E i n i = 1 SIGCOMM '05 14
CDF of FB Prediction Error Overestimation by >100% (E>1) for 40% of the measurements Dominance of overestimation errors (E>0) Prevalent occurrences of T’ < T and p’ < p CDF of Relative Prediction Error (lossy paths) 100 80 60 CDF (%) 40 20 0 -4 -2 0 2 4 6 8 10 Relative Error E SIGCOMM '05 15
CDF of FB Prediction Error Overestimation by >100% (E>1) for 40% of the measurements Dominance of overestimation errors (E>0) Prevalent occurrences of T’ < T and p’ < p Loss Rate Increase 0.001 0.02 0.04 0.06 0.08 0.1 100 80 60 CDF (%) Loss rate increase RTT increase 40 20 0 0 20 40 60 80 100 RTT Increase (ms) SIGCOMM '05 16
Errors Due to Sampling Differences Prediction using Ping RTT & loss rate measurements during target flow Prediction errors are still significant, but overestimation & underestimation are almost symmetric CDF of Relative Prediction Error 100 RTT/loss rate during TCP flow RTT/loss rate prior to TCP flow 80 60 CDF (%) 40 20 0 -8 -6 -4 -2 0 2 4 6 8 Relative Error E SIGCOMM '05 17
Prediction Accuracy vs. Actual Throughput Large errors are more common in lower-throughput paths Explanation: in a congested path, slight load increase causes large loss rate increase SIGCOMM '05 18
Window-limited Flows Throughput is more predictable for window-limited TCP flows Explanation: window-limited flows do not saturate path’s bottleneck 80 W=20KB (window-limited) W=1MB (congestion-limited) 30 RMSRE (log scale) 10 2 1 0.5 0.1 0 5 10 15 20 Path Index SIGCOMM '05 19
Outline TCP throughput prediction: problem statement Formula-Based (FB) prediction A formula-based predictor Types of FB prediction errors Experimental evaluation History-Based (HB) prediction Typical history-based predictors Dealing with outliers and level shifts Experimental evaluation Predictability factors What makes some paths less predictable than others? SIGCOMM '05 20
History-Based Prediction General one-step forecasting problem ˆ = R f ( R , R ,...., R ) − n 1 2 n 1 We only consider simple linear predictors Moving Average (MA) 1 i ∑ ˆ + = X X i 1 k n = − + k i n 1 Exponentially Weighted Moving Average (EWMA) ˆ ˆ = α + − α X X ( 1 ) X + i 1 i i Non-seasonal Holt-Winters (HW) An EWMA variation that captures the time series trend ˆ ˆ ˆ = + X X X f s t i i i --smoothing ˆ ˆ = α + − α X X ( 1 ) X s f + i 1 i i ˆ ˆ ˆ ˆ = β − + − β --trend X ( X X ) ( 1 ) X t s s t + − − i 1 i i 1 i 1 SIGCOMM '05 21
Level Shifts (LS) and Outliers (OL) Why are LS and OL undesirable? Cause large prediction errors and differences among predictors; complicate the analysis of HB predictability Dealing with LS and OL is more important than choosing among predictors Actions: ignore OL, restart predictor upon LS UTAH-LULEA 18 16 14 Throughput (Mbps) 12 10 8 6 4 2 0 20 40 60 80 100 120 140 Measurement Epoch SIGCOMM '05 22
Overall HB Prediction Accuracy HB prediction is much more accurate than FB prediction 90% of traces have RMSREs < 0.4 (with LS/OL detection) With LS/OL detections, the choices of predictor and of predictor parameters make little difference CDF of Prediction Error (Holt-Winters) 100 80 60 CDF (%) 40 0.8-HW-LSO 0.8-HW 0.4-HW 20 0 0.01 0.1 0.4 1 5 RMSRE (log scale) SIGCOMM '05 23
Effect of Measurement Frequency Longer measurement period does not degrade accuracy significantly Even with single transfer every 24 minutes, RMSRE is below 0.4 in 75% of the traces CDF of Prediction Error vs. Sampling Frequency 100 80 3-min interval 6-min interval 24-min interval 45-min interval 60 CDF 40 20 0 0.01 0.1 0.2 0.4 1 5 RMSRE (log scale) SIGCOMM '05 24
Recommend
More recommend
