quickcheck
play

QuickCheck John Hughes DEMO Registry tests on multiple nodes - PowerPoint PPT Presentation

Jan 06, 2023 •989 likes •1.29k views

QuickCheck John Hughes DEMO Registry tests on multiple nodes 40000 35000 30000 Tests per minute 25000 20000 15000 10000 5000 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 Nodes Four core/8HT SkyLake i7

  1. QuickCheck John Hughes

  2. DEMO

  3. Registry tests on multiple nodes 40000 35000 30000 Tests per minute 25000 20000 15000 10000 5000 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 Nodes Four core/8HT SkyLake i7 Dual core/4HT Ivy Bridge i7

  4. Architecture Count tests, print dots, start and stop Master Worker Worker Worker Worker Generate and run tests

  5. How can the Master count the tests? Master Worker Worker Worker Worker

  6. How can the Master stop the workers? Master Worker Worker Worker Worker

  7. Benchmark Bouncer Mirror 1.2 million/second

  8. Multiple bouncers Bouncer Mirror Bouncer Mirror Bouncer Mirror

  9. Message round-trips per second Four core/8 thread i7 4500000 Round trips per second 4000000 3500000 3000000 2500000 2000000 1500000 1000000 500000 0 1 2 3 4 5 6 7 8 9 10 P--number of bouncers P bouncers, P mirrors

  10. Multiple bouncers, one mirror Bouncer Bouncer Mirror Bouncer

  11. Message round-trips per second Four core/8 thread i7 4500000 4000000 Round trips per second 3500000 3000000 2500000 2000000 1500000 1000000 500000 0 1 2 3 4 5 6 7 8 9 10 P--number of bouncers P bouncers, P mirrors P bouncers, 1 mirror

  12. Batch permissions? Master Worker Worker Worker Worker BUT this may delay termination!

  13. Alternative Architecture Master Manager Manager Manager Manager Yes Another? Worker Worker Worker Worker

  14. • Every worker • Stopping can be communicates with its slightly delayed own manager— scalable!

  15. > eqc:quickcheck(examples:prop_reverse()). .................................................................................................... OK, passed 100 tests true > eqc:quickcheck(examples:prop_reverse()). ....................................................................................................(x10).(x1)......... OK, passed 119 tests true > eqc:quickcheck(eqc:in_parallel(examples:prop_reverse())). ....................................................................................................(x10)..(x1)...... OK, passed 126 tests true > eqc:quickcheck(eqc:on_nodes(examples:prop_reverse())). ....................................................................................................(x10)................................ ........................(x1)......... OK, passed 669 tests But how bad is it to run a few extra tests?

  16. What about node placement? Master Manager Manager Manager Manager Yes Another? Worker Worker Worker Worker

  17. What about node placement? Master Manager Manager Manager Manager Worker Worker Worker Worker

  18. What about node placement? Master Manager Manager Manager Manager Yes Another? Worker Worker Worker Worker

  19. P bouncers, 1 mirror, different nodes Four core/8 HT i7 160000 140000 120000 100000 80000 60000 40000 20000 0 1 2 3 4 5 6 7 8 9 10

  20. P bouncers, 1 mirror Four core/8HT i7 1600000 1400000 1200000 1000000 10-30x slower from a 800000 different node! 600000 400000 200000 0 1 2 3 4 5 6 7 8 9 10 Different nodes Same node

  21. P bouncers, 1 mirror Bouncers on dual core laptop, mirror on quad core 30000 25000 20000 15000 10000 5000 0 0 5 10 15 20 25 30 35

  22. Bounces per second (single bouncer) 1400000 1200000 1000000 800000 600000 400000 200000 0 Same LAN Same Host Same Node

  23. Bounces per second (single bouncer) 7 6 5 30x Log scale 4 50x 3 2 1 0 Same LAN Same Host Same Node

  24. What about success messages? Master Manager Manager Manager Manager Success Worker Worker Worker Worker

  25. Two-way vs one-way Bouncer Mirror 1.2 million/second 5.4 million/second

  26. Bounces/Messages per second 8 4x 10x 7 100x 6 Log scale 5 4 3 2 1 0 Same LAN Same Host Same Node Two way One way

  27. One more optimization… Send the total every There are a LOT of Master 100ms success messages Count them! Manager Manager Manager Manager Success Worker Worker Worker Worker

  28. • Tripled the speed of • Stopping can be even quickcheck(true)! more delayed

  29. Lessons • There is at least an order of magnitude difference between communication costs • Within a node • Between nodes • Between hosts • Latency is much worse affected than bandwidth • This affects design for performance • Favours asynchronous over synchronous communication between nodes • Optimising performance may require changes to observable behaviour • … and we didn’t even consider fault tolerance

Recommend

QuickCheck John Hughes Chalmers University/Quviq AB What is QuickCheck? A library for

QuickCheck John Hughes Chalmers University/Quviq AB What is QuickCheck? A library for

Specification Based Testing with QuickCheck John Hughes Chalmers University/Quviq AB What is QuickCheck? A library for writing and testing properties of program code Some code: A property: Properties as Code A test data A

1.08k views • 72 slides
Be#er Tes(ng with Less Work: QuickCheck Tes(ng in Prac(ce

Be#er Tes(ng with Less Work: QuickCheck Tes(ng in Prac(ce

Be#er Tes(ng with Less Work: QuickCheck Tes(ng in Prac(ce John Hughes QuickCheck in a Nutshell Test case Test case Test case Proper(es Test

820 views • 49 slides
QuickCheck: Beyond the Basics Dave Laing May 13, 2014 Basic QuickCheck Some properties

QuickCheck: Beyond the Basics Dave Laing May 13, 2014 Basic QuickCheck Some properties

QuickCheck: Beyond the Basics Dave Laing May 13, 2014 Basic QuickCheck Some properties reverse_involutive :: [Int] -> Bool reverse_involutive xs = xs == (reverse . reverse) xs sort_idempotent :: [Int] -> Bool sort_idempotent xs =

807 views • 38 slides
QuickCheck 11.11 The two boxes are on a frictionless surface. They had been sitting together at

QuickCheck 11.11 The two boxes are on a frictionless surface. They had been sitting together at

QuickCheck 11.11 The two boxes are on a frictionless surface. They had been sitting together at rest, but an explosion between them has just pushed them apart. How fast is the 2 kg box going? A. 1 m/s B. 2 m/s C. 4 m/s D. 8 m/s

834 views • 4 slides
QuickCheck 11.2 A 2.0 kg object moving to the right with speed 0.50 m/s experiences the force

QuickCheck 11.2 A 2.0 kg object moving to the right with speed 0.50 m/s experiences the force

QuickCheck 11.2 A 2.0 kg object moving to the right with speed 0.50 m/s experiences the force shown. What are the objects speed and direction after the force ends? A. 0.50 m/s left B. At rest C. 0.50 m/s right D. 1.0 m/s right E. 2.0 m/s

178 views • 4 slides
QuickCheck 24.12 A cylindrical Gaussian surface surrounds an infinite line of charge. The flux

QuickCheck 24.12 A cylindrical Gaussian surface surrounds an infinite line of charge. The flux

QuickCheck 24.12 A cylindrical Gaussian surface surrounds an infinite line of charge. The flux e through the two flat ends of the cylinder is A. 0 2 2 rE B. 2 r 2 E C. 2 rLE D. E. It will require an integration to find

697 views • 6 slides
Ch 22 HW due Friday, January 17 @ 23:59 QuickCheck 23.2 What is the direction of the

Ch 22 HW due Friday, January 17 @ 23:59 QuickCheck 23.2 What is the direction of the

Ch 22 HW due Friday, January 17 @ 23:59 QuickCheck 23.2 What is the direction of the electric field at the dot? E. The field is zero. QuickCheck 23.2 What is the direction of the electric field at the dot? E. The field is zero.

246 views • 7 slides
QuickCheck 4.2 A heavy red ball is released from rest 2.0 m above a flat, horizontal surface. At

QuickCheck 4.2 A heavy red ball is released from rest 2.0 m above a flat, horizontal surface. At

QuickCheck 4.2 A heavy red ball is released from rest 2.0 m above a flat, horizontal surface. At exactly the same instant, a yellow ball with the same mass is fired horizontally at 3.0 m/s. Which ball hits the ground first? A. The red ball

1.01k views • 4 slides
Quiz #2 Friday, March 20 QuickCheck 28.14 The three bulbs are identical and the two

Quiz #2 Friday, March 20 QuickCheck 28.14 The three bulbs are identical and the two

Ch 26 HW due Friday, March 6 (today) @ 23:59 Quiz #2 Friday, March 20 QuickCheck 28.14 The three bulbs are identical and the two batteries are identical. Compare the brightnesses of the bulbs. A B C A. A C B B. C.

338 views • 7 slides
Graphical editing support for QuickCheck models Thomas Arts, Kirill Bogdanov, Alex Gerdes, John

Graphical editing support for QuickCheck models Thomas Arts, Kirill Bogdanov, Alex Gerdes, John

Graphical editing support for QuickCheck models Thomas Arts, Kirill Bogdanov, Alex Gerdes, John Hughes This project has received funding from the EU FP7 Collaborative project PROWESS , grant number 317820, http://www.prowessproject.eu

127 views • 12 slides
Testing monadic programs using QuickCheck and state machine based models Stevan Andjelkovic

Testing monadic programs using QuickCheck and state machine based models Stevan Andjelkovic

Testing monadic programs using QuickCheck and state machine based models Stevan Andjelkovic 2018.2.23, BOBKonf (Berlin) Introduction Property based testing in general How to apply property based testing to stateful and concurrent

474 views • 18 slides
due Friday, January 31 @ 23:59 Quiz #1 Friday, January 31 QuickCheck 24.8 Which spherical

due Friday, January 31 @ 23:59 Quiz #1 Friday, January 31 QuickCheck 24.8 Which spherical

Ch 24 HW due Friday, January 31 @ 23:59 Quiz #1 Friday, January 31 QuickCheck 24.8 Which spherical Gaussian surface has the larger electric flux? A. Surface A B. Surface B C. They have the same flux. D. Not enough information

379 views • 8 slides
QuickCheck 10.2 Starting from rest, a marble first rolls down a steeper hill, then down a less

QuickCheck 10.2 Starting from rest, a marble first rolls down a steeper hill, then down a less

QuickCheck 10.2 Starting from rest, a marble first rolls down a steeper hill, then down a less steep hill of the same height. For which is it going faster at the bottom? A. Faster at the bottom of the steeper hill. B. Faster at the bottom

352 views • 4 slides
Suppose 1. we wanted to design a machine architecture

Suppose 1. we wanted to design a machine architecture

Test-Driven Development of an Informa3on-Flow ISA A QuickCheck Adventure Arthur Azevedo de Amorim, Catalin Hritcu, John Hughes, Leonidas Lampropoulos, Ulf

684 views • 37 slides
5 7 pm in ACS 130 Midterm Friday, February 14 No tutorials this week No

5 7 pm in ACS 130 Midterm Friday, February 14 No tutorials this week No

QSCU Midterm Review session Wednesday, February 12 (today) 5 7 pm in ACS 130 Midterm Friday, February 14 No tutorials this week No homework this week QuickCheck 26.1 This is a graph of the x -component of the

201 views • 7 slides
Optional Ch. 25 28 review problems on Mastering Physics Quiz #2 Friday, March 20

Optional Ch. 25 28 review problems on Mastering Physics Quiz #2 Friday, March 20

Ch 27 HW due Friday, March 13 (today) @ 23:59 Optional Ch. 25 28 review problems on Mastering Physics Quiz #2 Friday, March 20 QuickCheck 28.19 The capacitor is initially unchanged. Immediately after the switch closes, the

305 views • 7 slides
QSCU Midterm Review session Wednesday, February 12 5 7 pm in ACS 130 Midterm

QSCU Midterm Review session Wednesday, February 12 5 7 pm in ACS 130 Midterm

Ch 24 HWb due Friday, February 7 (today) @ 23:59 QSCU Midterm Review session Wednesday, February 12 5 7 pm in ACS 130 Midterm Friday, February 14 QuickCheck 25.2 Two positive charges are equal. Which has more

78 views • 5 slides
2013-11-19: Test Data Generators Peter Thiemann 19 November 2013 1 Monads: An interface for

2013-11-19: Test Data Generators Peter Thiemann 19 November 2013 1 Monads: An interface for

2013-11-19: Test Data Generators Peter Thiemann 19 November 2013 1 Monads: An interface for instructions 1.1 primitive instructions 1.2 combining instructions (bind) 1.3 injecting values (return) 2 QuickCheck Instructions 2.1

560 views • 8 slides
Labs start this week Do your pre-lab assignments Tutorials start this week Quiz #1

Labs start this week Do your pre-lab assignments Tutorials start this week Quiz #1

Ch 23 HW due Friday, January 24 @ 23:59 Labs start this week Do your pre-lab assignments Tutorials start this week Quiz #1 Friday, January 31 QuickCheck 23.9 Two protons, A and B, are next to an infinite plane of

399 views • 5 slides

More recommend