parallel func onal arrays
play

Parallel Func+onal Arrays Ananya Kumar Guy Blelloch Robert Harper - PowerPoint PPT Presentation

Jan 20, 2023 •223 likes •820 views

Parallel Func+onal Arrays Ananya Kumar Guy Blelloch Robert Harper Carnegie Mellon University Goals Func+onal arrays Efficient (constant +me) Parallel Well defined cost seman+cs Previous Work - Monads Thread mutable state

  1. Parallel Func+onal Arrays Ananya Kumar Guy Blelloch Robert Harper Carnegie Mellon University

  2. Goals • Func+onal arrays • Efficient (constant +me) • Parallel • Well defined cost seman+cs

  3. Previous Work - Monads • Thread mutable state • Enforce single reference to array • Need completely different code • Not parallel

  4. Previous Work – Specialized Type System • Enforce single threadedness of arrays • Not available in most languages • Hard to reason about

  5. Previous Work – Reference Coun+ng • Check reference counts • If one, update in place, else copy • Depends on compiler • Hard to reason about

  6. Sequences A = NEW(5, 0) 0 0 0 0 0 B = SET(A, 0, 3) C = SET(A, 2, 3) 3 0 0 0 0 0 0 3 0 0 D = SET(C, 4, 14) 0 0 3 0 14 E = SET(D, 1, 11) 0 11 3 0 14

  7. Sequences A = NEW(5, 0) 0 0 0 0 0 B = SET(A, 0, 3) C = SET(A, 2, 3) 3 0 0 0 0 0 0 3 0 0 D = SET(C, 4, 14) 0 0 3 0 14 E = SET(D, 1, 11) 0 11 3 0 14

  8. Sequences A = NEW(5, 0) 0 0 0 0 0 B = SET(A, 0, 3) C = SET(A, 2, 3) 3 0 0 0 0 0 0 3 0 0 D = SET(C, 4, 14) 0 0 3 0 14 E = SET(D, 1, 11) 0 11 3 0 14

  9. Sequences A = NEW(5, 0) 0 0 0 0 0 B = SET(A, 0, 3) C = SET(A, 2, 3) 3 0 0 0 0 0 0 3 0 0 D = SET(C, 4, 14) 0 0 3 0 14 E = SET(D, 1, 11) 0 11 3 0 14

  10. Sequences A = NEW(5, 0) 0 0 0 0 0 B = SET(A, 0, 3) C = SET(A, 2, 3) 3 0 0 0 0 0 0 3 0 0 D = SET(C, 4, 14) 0 0 3 0 14 E = SET(D, 1, 11) 0 11 3 0 14

  11. Sequences A = NEW(5, 0) 0 0 0 0 0 B = SET(A, 0, 3) C = SET(A, 2, 3) 3 0 0 0 0 0 0 3 0 0 D = SET(C, 4, 14) 0 0 3 0 14 E = SET(D, 1, 11) 0 11 3 0 14

  12. Sequences A = NEW(5, 0) 0 0 0 0 0 B = SET(A, 0, 3) C = SET(A, 2, 3) 3 0 0 0 0 0 0 3 0 0 D = SET(C, 4, 14) 0 0 3 0 14 E = SET(D, 1, 11) 0 11 3 0 14

  13. Sequences A = NEW(5, 0) 0 0 0 0 0 B = SET(A, 0, 3) C = SET(A, 2, 3) 3 0 0 0 0 0 0 3 0 0 D = SET(C, 4, 14) 0 0 3 0 14 E = SET(D, 1, 11) 0 11 3 0 14

  14. Sequences A = NEW(5, 0) 0 0 0 0 0 B = SET(A, 0, 3) C = SET(A, 2, 3) 3 0 0 0 0 0 0 3 0 0 D = SET(C, 4, 14) 0 0 3 0 14 E = SET(D, 1, 11) 0 11 3 0 14

  15. Sequences A = NEW(5, 0) 0 0 0 0 0 B = SET(A, 0, 3) C = SET(A, 2, 3) 3 0 0 0 0 0 0 3 0 0 D = SET(C, 4, 14) 0 0 3 0 14 E = SET(D, 1, 11) 0 11 3 0 14

  16. Previous Work • N = size of array • Dietz – O(log log N) per opera+on • Trailer arrays – O(1) for leaves • Improvements by Chuang, O’ Neill • No support for concurrency

  17. Our Approach • Func+onal • Efficient – O(1) for leaves, fast for interior • Parallel – wait-free • Well defined cost seman+cs

  18. Sequence Implementa+on C 0 11 3 0 14 2 D 3 E 4

  19. Main Sec+ons • Cost dynamics • Concurrent implementa+on

  20. Fork-Join Parallelism (1+2) || (3+4)

  21. Fork-Join Parallelism (1+2) || (3+4) Fork

  22. Fork-Join Parallelism (1+2) || (3+4) 3+4 1+2

  23. Fork-Join Parallelism (1+2) || (3+4) 3+4 1+2 7 3

  24. Fork-Join Parallelism (1+2) || (3+4) 3+4 1+2 7 3 Join

  25. Fork-Join Parallelism (1+2) || (3+4) 3+4 1+2 7 3 (3, 7)

  26. Work and Span N log(N) 1 Work: size of cost tree Span: depth of cost tree 1 1 1

  27. Work and Span N log(N) 1 Work: N + log(N) + 4 Span: N + log(N) + 2 1 1 1

  28. Scheduling Theorems • Work + Span gives execu+on cost on P processor machine • Goal: evaluate cost of using sequences on a P processor machine • Sufficient to evaluate work and span

  29. Parallel Structural Dynamics • Cost of running program with ∞ processors • Determinis+c

  30. Interleaved Structural Dynamics • Cost of running program with 1 processor • Non-determinis+c

  31. Interleaved Structural Dynamics • Store which sequences are interior and leaf

  32. Work = Non-Determinis+c A (leaf), size N GET GET GET SET Join

  33. Work (Good Interleaving) A (leaf), size N GET GET Current Work: 1 Total Work: 1 GET SET Join

  34. Work (Good Interleaving) A (leaf), size N GET GET Current Work: 1 Total Work: 2 GET SET Join

  35. Work (Good Interleaving) A (leaf), size N GET GET Current Work: 1 Total Work: 3 GET SET Join

  36. Work (Good Interleaving) A (leaf), size N GET GET Current Work: 1 Total Work: 4 GET SET Join

  37. Work = Non-Determinis+c A (leaf), size N GET GET GET SET Join

  38. Work (Bad Interleaving) A (leaf), size N GET GET Current Work: 1 Total Work: 1 GET SET Join

  39. Work (Bad Interleaving) A (leaf), size N GET GET Current Work: 1 Total Work: 2 GET SET Join

  40. Work (Bad Interleaving) A (leaf), size N GET GET Current Work: log(N) Total Work: 2 + log(N) GET SET Join

  41. Work (Bad Interleaving) A (leaf), size N GET GET Current Work: log(N) Total Work: 2 + 2log(N) GET SET Join

  42. GET-GET Case A (leaf), size N GET GET GET GET Join

  43. SET-GET Case A (leaf), size N GET GET GET SET Join

  44. SET-SET Case A (leaf), size N SET GET GET SET Join

  45. Upper Bounding Work • Determinis+c evalua+onal dynamics • Store which sequences are leaf and interior • Store the number of “cheap” (cost = 1) GETs on each sequence • At the join, if sequence was modified on one side, make the GETs expensive (cost = log(N))

  46. Upper Bounding Work • Showed that upper bounds are valid for all inter-leavings • Showed that the upper bound is +ght *

  47. A = NEW(5, 0) Seq A ArrayData 1 (Version = 1) Version 1 0 0 0 0 0

  48. B = SET(A, 2, 5) Seq A ArrayData 1 (Version = 2) Version 1 0 0 5 0 0 Seq B Version 1 Version 2 Value 0

  49. Naïve SET • Implementa+on of SET(A, i, v) • First set values[i] = v • Then add a log entry to arraydata

  50. GET-SET Race Sequence A, version 1 Array data AD, version 1 Values = [0, 0, 0, 0, 0] Logs = empty Thread 1 Thread 2 Result Step 1 Values[2] = 5 Step 2 GET(A, 2) Step 3 Add log entry to Logs[i]

  51. GET-SET Race Sequence A, version 1 Array data AD, version 1 Values = [0, 0, 0, 0, 0] Logs = empty Thread 1 Thread 2 Result Step 1 Values[2] = 5 ✓ Step 2 GET(A, 2) Step 3 Add log entry to Logs[i]

  52. GET-SET Race Sequence A, version 1 Array data AD, version 1 Values = [0, 0, 0, 0, 0] Logs = empty Thread 1 Thread 2 Result Step 1 Values[2] = 5 ✓ Step 2 GET(A, 2) 5 Step 3 Add log entry to Logs[i]

  53. GET-SET Race Sequence A, version 1 Array data AD, version 1 Values = [0, 0, 0, 0, 0] Logs = empty Thread 1 Thread 2 Result Step 1 Values[2] = 5 ✓ Step 2 GET(A, 2) 5 Step 3 Add log entry to ✓ Logs[i]

  54. A Wait-Free Solu+on • Can be fixed by adding log entry before muta+ng values array • Other issues in GET require careful ordering • Other issues in SET require compare & swap

  55. Experimental Results • Compared sequences to regular arrays • Random & sequen+al accesses • Wri+ng: 2-3 +mes slower • Reading: under 10% slower

  56. Concurrent Results • Compared – 1 thread reading million +mes – 2 threads reading half million +mes • 2 threads were > 1.75 +mes faster

  57. Summary • Func+onal array implementa+on • O(1) opera+ons for leaf • Wait-free concurrent • Well defined cost seman+cs

  58. Future Work • Prove concurrent costs of sequence implementa+on • Tighter cost bounds • Extend to disjoint sets, unordered sets • Lower bound for func+onal array costs

  59. Acknowledgements • Joe Tassaror for lots of advice on correctness proof • Danny Sleator for ideas on lower bounds for func+onal array costs • NSF, Air Force Office, Intel for grants

Recommend

Parallel Func+onal Programming Lecture 3 Mary Sheeran with

Parallel Func+onal Programming Lecture 3 Mary Sheeran with

Parallel Func+onal Programming Lecture 3 Mary Sheeran with thanks to Simon Marlow for use of slides and to Koen Claessen for the guest appearance

628 views • 43 slides
Parallel Func+onal Programming Lecture 2 Mary Sheeran (with

Parallel Func+onal Programming Lecture 2 Mary Sheeran (with

Parallel Func+onal Programming Lecture 2 Mary Sheeran (with thanks to Simon Marlow for use of slides) h>p://www.cse.chalmers.se/edu/course/pfp Remember

1.18k views • 83 slides
Learn&Func*onal&Programming&with& PureScript (Or$I'll$buy$you$a$coffee!)

Learn&Func*onal&Programming&with& PureScript (Or$I'll$buy$you$a$coffee!)

Learn&Func*onal&Programming&with& PureScript (Or$I'll$buy$you$a$coffee!) John%A.%De%Goes%%@jdegoes Agenda Func&ons Types,.Kinds,.&.More.Func&ons FP.Toolbox OMG.COFFEE.BREAK!!!

1.48k views • 106 slides
Func%onal annota%on Uppsala 9th-11th may 2017 Lucile Soler Based on Jacques

Func%onal annota%on Uppsala 9th-11th may 2017 Lucile Soler Based on Jacques

Func%onal annota%on Uppsala 9th-11th may 2017 Lucile Soler Based on Jacques Dainat presenta%on Overview Func%onal annota%on Why? Understanding the func%on of gene

541 views • 35 slides
Func+on applica+ons (calls, invoca+ons) lambda denotes a anonymous func+on To use a func+on, you

Func+on applica+ons (calls, invoca+ons) lambda denotes a anonymous func+on To use a func+on, you

Racket Func+ons Func+ons: the most important building block in Racket (and 251) Func+ons/procedures/methods/subrou+nes abstract over computa+ons Func%ons in Racket Like Java methods & Python func+ons, Racket func+ons have arguments

523 views • 7 slides
Func%onal)Dependencies) Chapter)3) 1) 2) 3)

Func%onal)Dependencies) Chapter)3) 1) 2) 3)

Func%onal)Dependencies) Chapter)3) 1) 2) 3) O"en,&our&first&a.empts&at&DB&schemas&can&be& improved,&especially&by&elimina;ng& redundancy .& &

1.13k views • 51 slides
o.OM: Structured-Func<onal Communica<on between Computer Music

o.OM: Structured-Func<onal Communica<on between Computer Music

o.OM: Structured-Func<onal Communica<on between Computer Music Systems using OSC and Odot Jean Bresson, John MacCallum, Adrian Freed UC Berkeley Center

358 views • 20 slides
Func%onal reviews suppor%ng good governance in various

Func%onal reviews suppor%ng good governance in various

Func%onal reviews suppor%ng good governance in various sectors Dragos Dinu Romanian Centre for European Policies (CRPE) www.crpe.ro What do we discuss

281 views • 14 slides
Func%onal understanding of nutrient requirements for microbial growth

Func%onal understanding of nutrient requirements for microbial growth

Func%onal understanding of nutrient requirements for microbial growth Sef Heijnen, Department of Biotechnology, Faculty of Applied Sciences Growth by Division

185 views • 14 slides
Assimila'on of func'onal data and applica'on to ac've fires detec'on from satellites Jan Mandel

Assimila'on of func'onal data and applica'on to ac've fires detec'on from satellites Jan Mandel

Assimila'on of func'onal data and applica'on to ac've fires detec'on from satellites Jan Mandel University of Colorado Denver Ins5tute of Informa5cs, Czech Academy of Sciences With Ivan Kasanick, Adam K. Kochanski, Sher Schranz, and Mar5n

1.05k views • 47 slides
Func%onal Probabilis%c Programming CUFP 2013 Avi Pfeffer Charles

Func%onal Probabilis%c Programming CUFP 2013 Avi Pfeffer Charles

Func%onal Probabilis%c Programming CUFP 2013 Avi Pfeffer Charles River Analy2cs apfeffer@cra.com Outline What is probabilistic programming? History Our Figaro language

548 views • 31 slides
Func%onal Architecture for Social Intelligence October 22, 2014 Brian

Func%onal Architecture for Social Intelligence October 22, 2014 Brian

11/9/14 Func%onal Architecture for Social Intelligence October 22, 2014 Brian Klobucher Sam Spaulding Agenda Modularity Visual Pathways / Social PercepCon

194 views • 7 slides
Process Engineering for Func/onal Foods Ken Morison Department

Process Engineering for Func/onal Foods Ken Morison Department

Process Engineering for Func/onal Foods Ken Morison Department of Chemical and Process Engineering Biomolecular Interac/on Centre University of Canterbury

657 views • 47 slides
Differen'able Func'onal Programming Noel Welsh @noelwelsh underscore Goals Deep learning

Differen'able Func'onal Programming Noel Welsh @noelwelsh underscore Goals Deep learning

Differen'able Func'onal Programming Noel Welsh @noelwelsh underscore Goals Deep learning isnt deep Deep learning is func'onal programming You can implement it yourself Deep Learning Deep learning is supervised learning of

1.91k views • 161 slides
Javascript More powerful than you think Some tools for

Javascript More powerful than you think Some tools for

Master Informa-que - Universit Paris-Sud 2/7/15 Javascript More powerful than you think Some tools for building Func-onal: func-ons are

747 views • 7 slides
Javascript More powerful than you think Some tools for

Javascript More powerful than you think Some tools for

Master Informa-que - Universit Paris-Sud 1/15/14 Javascript More powerful than you think Some tools for building Func-onal: func-ons are

846 views • 5 slides
Func%ons Func%ons are groups of statements to which you

Func%ons Func%ons are groups of statements to which you

Func%ons Func%ons are groups of statements to which you give a name. Defining a func%on uses the " def " keyword. That group of

626 views • 27 slides
Module 2 Cosme-c Ingredients Natural Func-onal Raw Materials

Module 2 Cosme-c Ingredients Natural Func-onal Raw Materials

Module 2 Cosme-c Ingredients Natural Func-onal Raw Materials Module 2 Objec-ves Learn how ingredients are classified as natural Learn the different

1.23k views • 76 slides
IT Administra-ve Controls Week 3 Organizing an IT Func-on What func-ons are Opera-ons

IT Administra-ve Controls Week 3 Organizing an IT Func-on What func-ons are Opera-ons

IT Administra-ve Controls Week 3 Organizing an IT Func-on What func-ons are Opera-ons cri-cal to an IT Applica-on development organiza-on? Architecture Program management Security Compliance/quality Finance

245 views • 8 slides
Analy&c Window Fu Func&ons A prac'cal look at using analy'c func'ons Olympia Area

Analy&c Window Fu Func&ons A prac'cal look at using analy'c func'ons Olympia Area

Analy&c Window Fu Func&ons A prac'cal look at using analy'c func'ons Olympia Area SqlServer User Group By Gary Melhaff March 15, 2017 About me me Over 30 years in IT Cer'fica'ons & experience in Oracle, Teradata and Sql

406 views • 12 slides
Objec(ve Using func(ons Using modules Anima(on Jan 29, 2018 Sprenkle - CSCI111 1

Objec(ve Using func(ons Using modules Anima(on Jan 29, 2018 Sprenkle - CSCI111 1

Objec(ve Using func(ons Using modules Anima(on Jan 29, 2018 Sprenkle - CSCI111 1 Review What is a func(on? Jan 29, 2018 Sprenkle - CSCI111 2 1 Func(ons Input Output func(on (arguments) (return value) Argument list

379 views • 13 slides
Objectives: Discuss arrays Syntax Multi-dimensional arrays Arrays

Objectives: Discuss arrays Syntax Multi-dimensional arrays Arrays

Arrays Objectives: Discuss arrays Syntax Multi-dimensional arrays Arrays Definition Variables that store many objects of the same class. access elements using brackets [ ] Their indices start at 0 They are

812 views • 11 slides
PHP Arrays Arrays in PHP are quite versa1le See

PHP Arrays Arrays in PHP are quite versa1le See

PHP Arrays Arrays in PHP are quite versa1le See h5p://php.net/manual/en/language.types.array.php We can use them as we use tradi1onal arrays, indexing

387 views • 20 slides
A Parallel Numerical Solver Using Hierarchically Tiled Using Hierarchically Tiled Arrays James

A Parallel Numerical Solver Using Hierarchically Tiled Using Hierarchically Tiled Arrays James

A Parallel Numerical Solver Using Hierarchically Tiled Using Hierarchically Tiled Arrays James Brodman, G. Carl Evans, Murat Manguoglu, Ahmed Sameh, Mara J. g g Garzarn, and David Padua Outline Motivation Hierarchically Tiled

868 views • 34 slides

More recommend