λ λ CS 251 Fall 2019 CS 251 Fall 2019 Parallelism and Concurrency in 251 Principles of Programming Languages Principles of Programming Languages Ben Wood Ben Wood • Goal: encounter – essence, key concerns Parallelism – non-sequential thinking – some high-level models – some mid-to-high-level mechanisms (and Concurrency) • Non-goals: – performance engineering / measurement – deep programming proficiency – exhaustive survey of models and mechanisms https://cs.wellesley.edu/~cs251/f19/ 1 Parallelism 2 Parallelism Pa Parallelism Concu Co curre rrency cy Coordinate access Use more resources to shared resources. to complete work faster. Eliminate 1 big assumption: workers = computations data / work Evaluation happens di divide ded d among as a sequence of ordered steps. sh share workers = resources data = resources Both can be expressed using a variety of primities. Parallelism 3 Parallelism 4
Manticore Manticore Parallel Arrays: 'a parray Parallelism via Manticore • Extends SML with language features for literal parray [| e1 , e2 , …, en |] parallelism/concurrency. • Mix research vehicle / established models. • Parallelism patterns: integer ranges [| elo to ehi by estep |] – data parallelism: • parallel arrays • parallel tuples parallel mapping comprehensions [| e | x in elems |] – task parallelism: • parallel bindings • parallel case expressions parallel filtering comprehensions [| e | x in elems where pred |] • Unifying model: – futures / tasks • Mechanism: – work-stealing Parallelism 5 Parallelism 6 Manticore parallel array comprehensions Data Parallelism many argument data of same type [| e1 | x in e2 |] Evaluation rule: 1. Under the current environment, E, evaluate parallelize pa ze e2 to a parray v2 . ap appli licat cation o of s same ame o operat ation to to all data 2. For each element vi in v2 , wi with no o con onstra raint on re on relative timing or order : no ordering/ interdependence 1. Create new environment Ei = x ↦ vi, E . 2. Under environment Ei , evaluate e1 to a value vi' 3. The result is [| v1', v2', …, vn' |] many result data of same type Parallelism 7 Parallelism 8
Manticore Manticore Parallel Map / Filter Parallel Reduce fun reduceP f init xs = … fun mapP f xs = [| f x | x in xs |] : (('a * 'a) -> 'a) -> 'a -> 'a parray -> 'a sibling of fold : ('a -> 'b) -> 'a parray -> 'b parray f must be associative f ( , ) fun filterP p xs = [| x | x in xs where p x |] f ( , ) f ( , ) : ('a -> bool) -> 'a parray -> 'a parray f ( , ) f ( , ) f ( , ) f ( , ) … … … … Parallelism 9 Parallelism 10 Manticore pivot Task Parallelism Parallel Bindings sorted_lt sorted_eq sorted_gt concatP fun qsortP (a: int parray) : int parray = if lengthP a <= 1 then a else let val pivot = a ! 0 (* parray indexing *) parallelize pa ze appl pplication on Start ev St evaluating pval sorted_lt = of different ope of operation ons in p in parallel el qsortP (filterP (fn x => x < pivot) a) within larg wi rger r computation but bu pval sorted_eq = do don’t wait filterP (fn x => x = pivot) a unt until il need needed ed. some ordering/interdependence pval sorted_gt = controlled explicitly qsortP (filterP (fn x => x > pivot) a) in concatP ( sorted_lt , concatP ( sorted_eq , sorted_gt )) end Wa Wait until results are ready before using them. Parallelism 11 Parallelism 12
future = promise speculatively forced in parallel Manticore Parallel Cases Futures: unifying model for Manticore parallel features signature FUTURE = datatype 'a bintree = Empty sig | Node of 'a * 'a bintree * 'a bintree type 'a future fun find_any t e = (* Produce a future for a thunk. case t of Like Promise.delay . *) Empty => NONE Ev Eval aluat uate these in n par aral allel. val future : (unit -> ’a) -> ’a future | Node (elem, left, right) => if e = elem then SOME t (* Wait for the future to complete and return the result. else Like Promise.force . *) pcase find_any left e & find_any right e of val touch : ’a future -> ’a If If o one ne f finis inishes w wit ith S SOME, r , retur urn it n it without waiting for wi r the other. (* More advanced features. *) datatype 'a result = VAL of 'a | EXN of exn SOME tree & ? => SOME tree | ? & SOME tree => SOME tree (* Check if the future is complete and get result if so. *) val poll : ’a future -> ’a result option | NONE & NONE => NONE (* Stop work on a future that won't be needed. *) val cancel : ’a future -> unit If If b both f finis inish w wit ith N NONE, r , retur urn N n NONE. Parallelism 13 Parallelism 14 end Futures: timeline visualization 1 Futures: timeline visualization 2 time time let let val f = future (fn () => e) val f = future (fn () => e) in in work work … … (touch f) (touch f) … … end end Parallelism 15 Parallelism 16
pval as future sugar Parray ops as futures: rough idea 1 Suppose we represent parrays as lists* of of elements : let pval x = e in … x … end [| f x | x in xs |] let val x = future (fn () => e ) in … (touch x ) … end map touch ( map (fn x => *a bit more: implicitly cancel an untouched future future (fn () => f x )) once it becomes clear it won't be touched. xs) *not the actual implementation Parallelism 17 Parallelism 18 Odds and ends Parray ops as futures: rough idea 2 Suppose we represent parrays as lists* of of element futures : • pcase: not just future sugar – Choice is a distinct primitive* not offered by [| f x | x in xs |] futures alone. • Where do execution resources from futures come from? How are they managed? • Tasks vs futures: map (fn x => future (fn () => f (touch x) )) – function calls vs. val bindings. xs • Forward to concurrency and events… Ke Key semantic difference 1 vs 2? *at least when implemented well. *not the actual implementation Parallelism 19 Parallelism 20
Recommend
More recommend
