A Performance Perspective on Web Optimized Protocol Stacks: - PowerPoint PPT Presentation

A Performance Perspective on Web Optimized Protocol Stacks: TCP+TLS+HTTP/2 vs. QUIC Konrad Wolsing, Jan Rüth, Klaus Wehrle, Oliver Hohlfeld https://comsys.rwth-aachen.de/ Aachen, 2019-07-22

Evolution of Web-stacks Multiplexing 2 0-RTT connection establishment No head-of-line blocking Evolvability over time (no ossifjcation) Multiplexing Encryption Loss Recovery Congestion Control Encryption H2 over gQUIC Loss Recovery Congestion Control Internet IP TCP TLS HTTP/2 UDP gQUIC Konrad Wolsing

Related Work (QUIC vs. TCP) 3 Konrad Wolsing

Related Work (QUIC vs. TCP) • Unoptimized TCP stacks 4 TCP+TLS+HTTP/2 and QUIC. evaluation on eye-to-eye level between Reproducible, user-centered performance Our Goal • User-centered metrics • Connection establishment performance Konrad Wolsing ◮ QUIC is optimized for web ◮ research focuses on PLT ◮ PLT is not suited for user perception ◮ TCP can be tuned too ◮ Related work does not tune TCP ◮ QUIC requires 0–1 RTT ◮ TCP+TLS usually 2 RTT

Achieving comparability • Protocol settings 5 QUIC, but with BBR as congestion control QUIC+BBR Google QUIC Version 43: IW32, Pacing, Cubic QUIC TCP+, but with BBR as congestion control TCP+BBR IW 32, Pacing, tuned network bufgers, no slow start after idle, Cubic TCP+ Stock TCP (Linux 4.18): IW10, Cubic TCP No head-of-line blocking • TCP + TLS + HTTP HTTP/2 No encryption overhead no 0-RTT connection setup Packet pacing Initial window 32 • QUIC Head-of-line blocking Use only HTTP/2 Use TLS1.3 without 0-RTT 1-RTT handshake Packet pacing Initial window 32 Konrad Wolsing

Testbed 262ms .468Mbps 1.89Mbps Downlink 25 Mbps 10.5 Mbps 3.54 Mbps .468Mbps 1.89Mbps RTT 24ms 74ms 94ms 761ms 2.8 Mbps Loss 0.0% 0.0% .048% 3.3% 6.0% Queue size 12ms 200ms 200ms 200ms 200ms 6 3.54 Mbps 5 Mbps MahiMahi Uplink 6.3.24.5 9.3.4.12 2.3.31.2 4.7.6.23 Delay Bandwidth Loss DNS statistics • Server speed test Konrad Wolsing MSS[7] DA2GC[7] 3G[1] LTE[2] DSL[2] • Network confjguration QUIC 10MB NGINX 10MB QUIC 1MB NGINX 1MB QUIC 10KB NGINX 10KB QUIC 2B NGINX 2B 0 100 200 300 400 500 Download time [ms]

Measurement • Replayed Websites 7 0.75s TCP X TCP performance gain TCP • Visual Metrics Konrad Wolsing Size [MB] ◮ 38 websites [8] from Alexa and Moz lists 0 2 4 6 telegraph.com wikipedia.org gnu.org wordpress.com phpbb.com w3.org ed.gov gravatar.com opera.com google.com gov.uk statcounter.com apache.org msn.com joomla.com spotify.com sciencedirect.com bit.ly imgur.com dotdash.com etsy.com researchgate.net nature.com academia.edu imdb.com QUIC = X QUIC − X TCP pinterest.com youtube.com intel.com demorgen.be harvard.edu facebook.com reddit.com sciencemag.org canvas.be github.com columbia.edu vtm.be nytimes.com 0 10 20 30 IPs [#] Visual Completeness in % ◮ Only PLT not above-the-fold 100 0 0 FVC 1 0.5s QUIC − 0.75s TCP FVC SI 2 Time in s VC85 3 SI = -0.333 4 PLT VC85 PLT 5 LVC LVC 6

Performance gain DSL • QUIC outperforms even tuned TCP 8 • Steep curves: website size/structure has low impact Konrad Wolsing Network DSL FVC SI VC85 LVC PLT 1 . 0 CDF 0 . 5 0 . 0 − 0 . 5 0 . 0 − 0 . 5 0 . 0 − 0 . 5 0 . 0 − 0 . 5 0 . 0 − 0 . 5 0 . 0 (Downlink 25 Mbps, Uplink 5 Mbps, RTT 24 ms, Loss 0 . 0 % , [2]) • Tuning ⇒ increased performance

Performance gain LTE • Results similar to DSL 9 • But difgerences are slightly larger Konrad Wolsing Network LTE FVC SI VC85 LVC PLT 1 . 0 CDF 0 . 5 0 . 0 − 0 . 5 0 . 0 − 0 . 5 0 . 0 − 0 . 5 0 . 0 − 0 . 5 0 . 0 − 0 . 5 0 . 0 (Downlink 10 . 5 Mbps, Uplink 2 . 8 Mbps, RTT 74 ms, Loss 0 . 0 % , [2])

Performance gain 3G • Tuning impact reduced 10 • Variability rises Konrad Wolsing Network 3G FVC SI VC85 LVC PLT 1 . 0 CDF 0 . 5 0 . 0 − 0 . 5 0 . 0 − 0 . 5 0 . 0 − 0 . 5 0 . 0 − 0 . 5 0 . 0 − 0 . 5 0 . 0 (Downlink 3 . 54 Mbps, Uplink 3 . 54 Mbps, RTT 94 ms, Loss . 048 % , [1])

Performance gain DA2GC • Tuning becomes a coin toss 11 • QUIC performs still good Konrad Wolsing Network DA2GC FVC SI VC85 LVC PLT 1 . 0 CDF 0 . 5 0 . 0 − 0 . 5 0 . 0 − 0 . 5 0 . 0 − 0 . 5 0 . 0 − 0 . 5 0 . 0 − 0 . 5 0 . 0 (Downlink . 468 Mbps, Uplink . 468 Mbps, RTT 262 ms, Loss 3 . 3 % , [7]) ◮ No head-of-line blocking ◮ Larger SACK ranges

Performance gain MSS • Congestion control impacts performance 12 • QUIC with CUBIC still faster than TCP with CUBIC • BBR outperforms CUBIC Konrad Wolsing Network MSS FVC SI VC85 LVC PLT 1 . 0 CDF 0 . 5 0 . 0 − 0 . 5 0 . 0 − 0 . 5 0 . 0 − 0 . 5 0 . 0 − 0 . 5 0 . 0 − 0 . 5 0 . 0 (Downlink 1 . 89 Mbps, Uplink 1 . 89 Mbps, RTT 761 ms, Loss 6 . 0 % , [7])

Mean Performance Gain 13 Konrad Wolsing

Infmuence of Resources 14 Konrad Wolsing 3G (QUIC - TCP+) DA2GC (QUIC+BBR - TCP+BBR) MSS (QUIC+BBR - TCP+BBR) demorgen.be sciencemag.org vtm.be reddit.com canvas.be nytimes.com imdb.com columbia.edu msn.com researchgate.net youtube.com academia.edu facebook.com nature.com harvard.edu github.com intel.com etsy.com phpbb.com statcounter.com spotify.com imgur.com ed.gov gravatar.com dotdash.com pinterest.com w3.org apache.org joomla.com opera.com FVC gov.uk sciencedirect.com SI bit.ly wordpress.com VC85 gnu.org google.com wikipedia.org telegraph.com − 5 0 − 25 0 25 50 − 10 0 10 difference [s] difference [s] difference [s]

Discussing Metrics 15 Konrad Wolsing

Subtracting Design Difgerences 16 Konrad Wolsing 20 0 wikipedia.org gnu.org statcounter.com etsy.com w3.org phpbb.com telegraph.com gov.uk facebook.com nature.com github.com google.com msn.com columbia.edu ed.gov apache.org bit.ly joomla.com # of ips wordpress.com gravatar.com imdb.com reddit.com pinterest.com youtube.com researchgate.net opera.com sciencedirect.com harvard.edu imgur.com dotdash.com intel.com vtm.be academia.edu canvas.be spotify.com sciencemag.org demorgen.be nytimes.com

Subtracting Design Difgerences -529 wikipedia.org -1005 -3.834 Net Website [ms] [RTT] MSS gnu.org -1100 -1.447 MSS wikipedia.org -0.696 0.150 Congestion Control: BBR Net Website [ms] [RTT] MSS gnu.org -477 -0.628 MSS wikipedia.org 451 0.593 16 DA2GC 39 Mean difgerence under PLT with one subtracted RTT gnu.org Net Website [ms] [RTT] DSL gnu.org 1.6 0.066 DSL wikipedia.org -3.1 -0.128 LTE -30 gnu.org -0.344 DA2GC -0.570 -54 wikipedia.org 3G -32 -0.412 gnu.org 3G -0.175 -13 wikipedia.org LTE Konrad Wolsing QUIC − ( TCP+ − 1 · RTT )

Conclusion • Results 17 Thank you for your attention. Do users perceive QUIC as faster? especially on transport layer performance • Discussion more than protocol choice reduced connection establishment but the gap gets narrower Konrad Wolsing ◮ TCP tuning is not negligible ◮ QUIC is not build to primarily improve ◮ Still QUIC outperforms TCP, ◮ QUIC enables an evolvable stack ◮ QUIC mostly faster due to the RTT ◮ Open question: ◮ Congestion control sometimes matters

Literature I In Proceedings of the 30th Annual ACM Symposium on Applied Computing , pages 609–614. ACM, 2015. 18 In Proceedings of the Internet Measurement Conference (IMC) , pages 290–303. ACM, 2017. Taking a Long Look at QUIC: An Approach for Rigorous Evaluation of Rapidly Evolving Transport Protocols. A. M. Kakhki, S. Jero, D. Chofgnes, C. Nita-Rotaru, and A. Mislove. In International Conference on Communications (ICC) , pages 1–6. IEEE, May 2017. QUIC: Better for what and for whom? S. Cook, B. Mathieu, P. Truong, and I. Hamchaoui. HTTP over UDP: an Experimental Investigation of QUIC. P. Biswal and O. Gnawali. G. Carlucci, L. De Cicco, and S. Mascolo. Accessed: 2018-11-15. https://web.archive.org/web/20181115105855/https://breitbandmessung.de/interaktive-darstellung. Breitbandmessung Ergebnisse als interaktive Darstellung. Breitbandmessung. In Global Communications Conference (GLOBECOM) , pages 1–6. IEEE, 2016. Does quic make the web faster? Konrad Wolsing

Literature II Y. Yu, M. Xu, and Y. Yang. 19 Journal of Information Sciences and Computing Technologies , 7(1):673–682, 2018. Performance Evaluation of HTTP/2 over TLS+ TCP and HTTP/2 over QUIC in a Mobile Network. Y. Zheng, Y. Wang, M. Rui, A. Palade, S. Sheehan, and E. O’Nuallain. In 36th International Performance Computing and Communications Conference (IPCCC) , pages 1–8. IEEE, Dec 2017. When QUIC meets TCP: An experimental study. In Proceedings of the World Wide Web Conference (WWW) , pages 1755–1764. ACM, 2018. P. Megyesi, Z. Krämer, and S. Molnár. HTTP/2 Prioritization and its Impact on Web Performance. M. Wijnants, R. Marx, P. Quax, and W. Lamotte. In Proceedings of the World Wide Web Conference on World Wide Web (WWW) , pages 1449–1458, 2018. Mile High WiFi: A First Look At In-Flight Internet Connectivity. J. P. Rula, J. Newman, F. E. Bustamante, A. M. Kakhki, and D. Chofgnes. In 2016 IEEE International Conference on Communications (ICC) , pages 1–6. IEEE, 2016. How quick is QUIC? Konrad Wolsing