Measuring Performance on OpenBSD Alexander Bluhm bluhm@openbsd.org - PowerPoint PPT Presentation

What did exist before? How does it work? What are the findings? What is the Conclusion? Measuring Performance on OpenBSD Alexander Bluhm bluhm@openbsd.org EuroBSDCon, September 2019

What did exist before? How does it work? What are the findings? What is the Conclusion? Agenda What did exist before? 1 How does it work? 2 What are the findings? 3 What is the Conclusion? 4

What did exist before? How does it work? What are the findings? What is the Conclusion? genua Firewall Testbed HPF

What did exist before? How does it work? What are the findings? What is the Conclusion? Multi User, Multi Purpose Hardware Setup 1 Gbit 1 Gbit Source 1 Switch Drain 1 Source 2 Drain 2 . . . . . . Target 1 Target 2 Source n Drain n . . . 10 Gbit 10 Gbit Target n

What did exist before? How does it work? What are the findings? What is the Conclusion? Performance Hardware Design 10 Gbit Target Target

What did exist before? How does it work? What are the findings? What is the Conclusion? Existing Regression Tests

What did exist before? How does it work? What are the findings? What is the Conclusion? Regression History for i386 history severity

What did exist before? How does it work? What are the findings? What is the Conclusion? Agenda What did exist before? 1 How does it work? 2 What are the findings? 3 What is the Conclusion? 4

What did exist before? How does it work? What are the findings? What is the Conclusion? Performance Goals history reproducable details available drill down automatic

What did exist before? How does it work? What are the findings? What is the Conclusion? Performance History advance by step install collect release results run checkout compile tests at date kernel

What did exist before? How does it work? What are the findings? What is the Conclusion? Performance Tests Overview cvs run install checkout date version steps run history run detail test command

What did exist before? How does it work? What are the findings? What is the Conclusion? Performance Run at Date checkout build kernel date quirks commits run and install log average numbers unstable results

What did exist before? How does it work? What are the findings? What is the Conclusion? Performance Repeat at CVS Checkout repeat standard outlier count deviation reboot log single result

What did exist before? How does it work? What are the findings? What is the Conclusion? Weekly from 6.2 to 6.3 TCP Performance A C B D E F G H I J 4 . 5 × 10 9 4 × 10 9 3 . 5 × 10 9 3 × 10 9 2 . 5 × 10 9 bits/sec 2 × 10 9 1 . 5 × 10 9 1 × 10 9 5 × 10 8 0 2017-11-01 2017-12-01 2018-01-01 2018-02-01 2018-03-01 Checkout (date)

What did exist before? How does it work? What are the findings? What is the Conclusion? Quirks from 6.2 to 6.3 A OpenBSD/amd64 6.2 release B fix cvs vendor branch checkout C clang update LLVM to 5.0.0 D pfctl pf packet rate matching E move kernel source file dwiic.c F pfctl pf divert type G sysctl struct vfsconf H clang update LLVM to 5.0.1 I pfctl pf syncookies J OpenBSD/amd64 6.3 release

What did exist before? How does it work? What are the findings? What is the Conclusion? Build Quirks advance build checkout by step toolchain userland run checkout compile tests at date kernel

What did exist before? How does it work? What are the findings? What is the Conclusion? Performance Hardware Target 1 Target 1 Target 2 Target 2 Linux

What did exist before? How does it work? What are the findings? What is the Conclusion? Performance Master web publish configure and compile master control remote local target target console install release

What did exist before? How does it work? What are the findings? What is the Conclusion? Agenda What did exist before? 1 How does it work? 2 What are the findings? 3 What is the Conclusion? 4

What did exist before? How does it work? What are the findings? What is the Conclusion? Drilldown from Week to Days TCP Performance 4 . 5 × 10 9 4 × 10 9 3 . 5 × 10 9 3 × 10 9 2 . 5 × 10 9 bits/sec 2 × 10 9 1 . 5 × 10 9 1 × 10 9 5 × 10 8 0 2017-12-28 2017-12-30 2018-01-01 2018-01-03 2018-01-05 2018-01-07 2018-01-09 Checkout (date)

What did exist before? How does it work? What are the findings? What is the Conclusion? Reproduce and Reboot run compile tests kernel run reboot relink again machine kernel

What did exist before? How does it work? What are the findings? What is the Conclusion? 6.5, 1 Day, 5 Tests, Keep Machine Running tpmr octeon unveil TCP Performance 3 . 5 × 10 9 3 . 4 × 10 9 3 . 3 × 10 9 bits/sec 3 . 2 × 10 9 3 . 1 × 10 9 3 × 10 9 2019-08-01 2019-08-01 2019-08-01 2019-08-01 2019-08-01 2019-08-01 2019-08-02 Checkout (date)

What did exist before? How does it work? What are the findings? What is the Conclusion? 6.5, 1 Day, 5 Tests, Reboot Machine tpmr octeon unveil TCP Performance 3 . 5 × 10 9 3 . 4 × 10 9 3 . 3 × 10 9 bits/sec 3 . 2 × 10 9 3 . 1 × 10 9 3 × 10 9 2019-08-01 2019-08-01 2019-08-01 2019-08-01 2019-08-01 2019-08-01 2019-08-02 Checkout (date)

What did exist before? How does it work? What are the findings? What is the Conclusion? 6.5, 1 Day, 5 Tests, Link and Reorder Kernel tpmr octeon unveil TCP Performance 3 . 5 × 10 9 3 . 4 × 10 9 3 . 3 × 10 9 bits/sec 3 . 2 × 10 9 3 . 1 × 10 9 3 × 10 9 2019-08-01 2019-08-01 2019-08-01 2019-08-01 2019-08-01 2019-08-01 2019-08-02 Checkout (date)

What did exist before? How does it work? What are the findings? What is the Conclusion? KARL Kernel Address Randomized Link obj2 obj1 shuffle obj4 main() obj3 gap random start() unmap locore0 boot

What did exist before? How does it work? What are the findings? What is the Conclusion? 6.5, 1 Day, 5 Tests, Sort Objects, Fixed Gap tpmr octeon unveil TCP Performance 3 . 5 × 10 9 3 . 4 × 10 9 3 . 3 × 10 9 bits/sec 3 . 2 × 10 9 3 . 1 × 10 9 3 × 10 9 2019-08-01 2019-08-01 2019-08-01 2019-08-01 2019-08-01 2019-08-01 2019-08-02 Checkout (date)

What did exist before? How does it work? What are the findings? What is the Conclusion? 6.5, 1 Day, 5 Tests, Sort Objects, Random Gap tpmr octeon unveil TCP Performance 3 . 5 × 10 9 3 . 4 × 10 9 3 . 3 × 10 9 bits/sec 3 . 2 × 10 9 3 . 1 × 10 9 3 × 10 9 2019-08-01 2019-08-01 2019-08-01 2019-08-01 2019-08-01 2019-08-01 2019-08-02 Checkout (date)

What did exist before? How does it work? What are the findings? What is the Conclusion? 6.5, 1 Day, 5 Tests, Align Sorted Objects, Fixed Gap tpmr octeon unveil TCP Performance 3 . 5 × 10 9 3 . 4 × 10 9 3 . 3 × 10 9 bits/sec 3 . 2 × 10 9 3 . 1 × 10 9 3 × 10 9 2019-08-01 2019-08-01 2019-08-01 2019-08-01 2019-08-01 2019-08-01 2019-08-02 Checkout (date)

What did exist before? How does it work? What are the findings? What is the Conclusion? Kernel Symbol Table nm bsd, diff, diffstat sort align +169 +13 unveil -169 -13 +25997 +28731 tpmr -25983 -28717

What did exist before? How does it work? What are the findings? What is the Conclusion? 6.4, 15 Days, 5 Tests, 2 CPU Sockets, Keep running TCP Performance Y Z ab c 4 . 5 × 10 9 4 × 10 9 3 . 5 × 10 9 3 × 10 9 2 . 5 × 10 9 bits/sec 2 × 10 9 1 . 5 × 10 9 1 × 10 9 5 × 10 8 0 2018-10-12 2018-10-14 2018-10-16 2018-10-18 2018-10-20 2018-10-22 2018-10-24 Checkout (date)

What did exist before? How does it work? What are the findings? What is the Conclusion? 2 CPU Sockets, Repeat, Keep running second cycle

What did exist before? How does it work? What are the findings? What is the Conclusion? from 6.2 to 6.3, 173 Days, Reorder TCP Performance A B C D E F G H I J 4 . 5 × 10 9 4 × 10 9 3 . 5 × 10 9 3 × 10 9 2 . 5 × 10 9 bits/sec 2 × 10 9 1 . 5 × 10 9 tx mitigation 1 × 10 9 5 × 10 8 0 2017-11-01 2017-12-01 2018-01-01 2018-02-01 2018-03-01 Checkout (date)

What did exist before? How does it work? What are the findings? What is the Conclusion? from 6.2 to 6.3, 173 Days, Make Kernel MAKE Performance A C B D E F G H I J 500 400 300 Meltdown sec 200 clang 5.0.0 clang 5.0.1 100 0 2017-11-01 2017-12-01 2018-01-01 2018-02-01 2018-03-01 Checkout (date)

What did exist before? How does it work? What are the findings? What is the Conclusion? from 6.3 to 6.4, 202 Days, Reorder TCP Performance K M L N O PQ RS TU V W X Y 4 . 5 × 10 9 4 × 10 9 3 . 5 × 10 9 3 × 10 9 2 . 5 × 10 9 bits/sec 2 × 10 9 1 . 5 × 10 9 retpoline witness witness 1 × 10 9 retguard 5 × 10 8 0 2018-04-01 2018-05-01 2018-06-01 2018-07-01 2018-08-01 2018-09-01 2018-10-01 Checkout (date)

What did exist before? How does it work? What are the findings? What is the Conclusion? from 6.3 to 6.4, 202 Days, Reorder MAKE Performance retpoline KL M N O PQ RS TU V W X Y 500 400 retguard 300 witness witness time DRM sec clang 6.0.0 200 100 0 2018-04-01 2018-05-01 2018-06-01 2018-07-01 2018-08-01 2018-09-01 2018-10-01 Checkout (date)