15-150 Fall 2020 Lecture 18 Sequences and parallelism Stephen - PowerPoint PPT Presentation

15-150 Fall 2020 Lecture 18 Sequences and parallelism Stephen Brookes

announcements • Election Day! • TAs will be posting lab solution videos (canvas, Thursday or Friday) • TAs offer NEW weekly REVIEW SESSION (Thursday, 6:30pm Pittsburgh time)

sequences signature SEQ = sig type 'a seq note the type of exception Range mapreduce val nth : int -> 'a seq -> 'a val length : 'a seq -> int val tabulate : (int -> 'a) -> int -> 'a seq val empty : unit -> 'a seq val map : ('a -> 'b) -> ('a seq -> 'b seq) val split : 'a seq -> 'a seq * 'a seq val reduce : ('a * 'a -> 'a) -> 'a -> 'a seq -> 'a val mapreduce : ('a -> 'b) -> ('b * 'b -> 'b) -> 'b -> 'a seq -> 'b end

• The SEQ signature can be implemented in many different ways • Each implementation has its own work/span characteristics • lists nth i S is O(n) • balanced trees nth i S is O(log n) • arrays nth i S is O(1)

comments • Last time: vector-based sequences • When length S = n > 1 and split S = (L, R) we had reduce g z S = g(reduce g z L, reduce g z R) • L and R have length ≈ n div 2 and we assumed g is constant-time, so we said W reduce (n) = 2W reduce (n div 2) + 1 • We forgot the work for split S, given as O(n) earlier, so we should have said W reduce (n) = 2W reduce (n div 2) + O(n) • The original answer is OK if split S has work O(1) Thanks to Sheng-Hsiang Sun for spotting this!

your task • Given a structure Seq : SEQ with known work/span characteristics • Design correct and efficient solutions to some parallelizable problems • prove correctness • calculate work and span

comments • We can also talk about our sequence operations abstractly , in a way that’s independent of the implementation • For example, just using SEQ functions: map f S = tabulate ( fn i => f(nth i S)) (length S) • Although map may not be defined this way in the Seq structure, the equation is valid (Both sides represent the same sequence) • But be careful: equal expressions may have different work , span (This is obvious, if you think about it!)

behavior empty( ) = ⟨ ⟩ length ⟨ v 0 ,…,v n-1 ⟩ = n if 0 ≤ i < n nth i ⟨ v 0 ,…,v n-1 ⟩ = v i tabulate f n = ⟨ f(0), …, f(n-1) ⟩

behavior empty( ) = ⟨ ⟩ length ⟨ v 0 ,…,v n-1 ⟩ = n if 0 ≤ i < n nth i ⟨ v 0 ,…,v n-1 ⟩ = v i tabulate f n = ⟨ f(0), …, f(n-1) ⟩ split ⟨ v 0 ,…,v n-1 ⟩ = ( ⟨ v 0 ,…,v m -1 ⟩ , ⟨ v m ,…,v n-1 ⟩ ) where m = n div 2

reduce fun reduce g z s = case (length s) of 0 => z | 1 => nth 0 s | _ => let val (s1, s2) = split s in g(reduce g z s1, reduce g z s2) end reduce g z ⟨ v 1 ,…,v n ⟩ = v 1 g v 2 … g v n when g is total & associative, z an identity for g

mapreduce fun mapreduce f g z s = case (length s) of 0 => z | 1 => f(nth 0 s) | _ => let val (s1, s2) = split s in g(mapreduce f g z s1, mapreduce f g z s2) end mapreduce f g z ⟨ v 1 ,…,v n ⟩ = (f v 1 ) g (f v 2 ) … g (f v n ) when g is total & associative, z an identity for g

lecture notes • I added a new section about associativity and identity elements • Remember that reduce and mapreduce should only be used with suitable g, z • The Lecture Notes include a proof of correctness for reduce • Shows why you need these properties!

lecture notes • I added a new section about associativity and identity elements • Remember that reduce and mapreduce should only be used with suitable g, z • The Lecture Notes include a proof of correctness for reduce • Shows why you need these properties! You should be reading the notes, too!

example reduce (op +) 0 ⟨ v 1, v 2 ⟩ = (op +) (reduce (op +) 0 ⟨ v 1 ⟩ , reduce (op +) 0 ⟨ v 2 ⟩ ) = (reduce (op +) 0 ⟨ v 1 ⟩ ) + (reduce (op +) 0 ⟨ v 2 ⟩ ) = (v 1 + 0) + (v 2 + 0) = v 1 + v 2 reduce g z behaves “correctly” when g is associative and z is an identity element reduce (op +) 0 ⟨ v 1, v 2 ⟩ = v 1 + v 2 + 0

example reduce (op +) 21 ⟨ v 1, v 2 ⟩ = (op +) (reduce (op +) 21 ⟨ v 1 ⟩ , reduce (op +) 21 ⟨ v 2 ⟩ ) = (reduce (op +) 21 ⟨ v 1 ⟩ ) + (reduce (op +) 21 ⟨ v 2 ⟩ ) = (v 1 + 21) + (v 2 + 21) = v 1 + v 2 + 42 reduce (op +) 21 ⟨ v 1, v 2 ⟩ ≠ v 1 + v 2 + 21

thinking abstractly • Use cost semantics to predict work and span of code • before testing or correctness analysis • Use behavioral specs to help us design correct code • Use inductive proofs to validate specs and confirm cost analysis

modular thinking • Don’t look inside the structure implementing SEQ • Just refer to the signature … • … and the specs

work/span Assume we have an implementation of SEQ with expression work span + nth i s O(1) O(1) same O(1) O(1) length s specs as tabulate f n O(n) O(1) before O(1) O(1) empty( ) map f s O(n) O(1) O(n) O(log n) reduce g z s O(n) O(log n) mapreduce f g z s O(1) O(1) split s when length of s is n, and f, g are constant time

gravitation • Newtonian laws • Simulate the motion of planets • for n bodies, this is O(n 2 ) work • Using sequences and parallel operations is very natural (!) • each body can calculate its step-by-step trajectory, independently • Will be faster than using lists and sequential evaluation

n bodies n 2 forces

Newton’s laws F = G m 1 m 2 / r 2 • Point masses attract each other with a force proportional to the product of the masses and the inverse square of the distance • Spherical bodies behave like point masses Newton, 1687

laws of motion Law 1 : If an object experiences no net force, its velocity is constant: - it moves in a straight line, with constant speed . Law 2 : The acceleration of a body is parallel and proportional to the net force acting on the body, and inversely proportional to the mass of the body, i.e., F = m a . Law 3 : When one body exerts a force F on a second body, the second body exerts an equal but opposite force − F on the first. Law 4 : There is no Law 4.

vectors Velocity, force and acceleration are vectors • Vectors have magnitude and direction speed = magnitude of velocity • Vectors can be added velocity + velocity = velocity acceleration + acceleration = acceleration • Vectors can be multiplied by a scalar scalar * velocity = velocity scalar * acceleration = acceleration

our version • 2-dimensional universe • Scalars are real numbers • Vectors are pairs of type real * real Easy to generalize...

bodies • A body has position, mass, and velocity • Positions are points , pairs of real numbers • A mass is a (positive) real number • A velocity is a 2D- vector • also represented as a pair of reals type point = real * real type vect = real * real type body = point * real * vect

vectors signature VECT = sig type vect = real * real val zero : vect val add : vect * vect -> vect val scale : real * vect -> vect val mag : vect -> real … end

structure Vect : VECT = struct type vect = real * real val zero = (0.0, 0.0) fun add ((x1, y1), (x2, y2)) = (x1+x2 , y1+y2) fun scale(c, (x, y)) = (c * x , c * y) fun mag (x, y) = Math.sqrt (x * x + y * y) end

points type point = real * real fun diff ((x1,y1):point, (x2,y2):point) : vect = (x2 - x1, y2 - y1) fun displace ((x,y):point, (x',y'):vect) : point = (x + x', y + y')

bodies ( mass, velocity ) position, type body = point * real * vect val sun = ((0.0,0.0), 332000.0, (0.0,0.0)) val earth = ((1.0, 0.0), 1.0, (0.0,18.0)) distance from sun to earth = one “ astronomical unit ” sun is 332000 times more massive the sun’s (relative) velocity is zero

motion • To calculate the motion of a body in a timestep • find the net acceleration due to other bodies • adjust the position and velocity of the body

accel accel : body -> body -> vect accel b 1 b 2 = use default of zero acceleration on b 1 when bodies are too close due to gravitational attraction of b 2

accel accel : body -> body -> vect accel b 1 b 2 = use default of zero acceleration on b 1 when bodies are too close due to gravitational attraction of b 2 fun accel (p 1 , _, _) (p 2 , m 2 , _) = let val d = diff(p 1 , p 2 ) val r = mag d in if r < 0.1 then zero else scale(G * m 2 /(r*r*r) , d) end

accel b 2 p 2 . m 2 b 1 Gm 2 /r 2 p 1 . m 1 r = distance from p 1 to p 2 = acceleration on b 1 due to b 2 Gm 2 /r 2

accel b 2 p 2 . m 2 b 1 Gm 1 /r 2 p 1 . m 1 r = distance from p 1 to p 2 = acceleration on b 2 due to b 1 G m 1 /r 2

accels accels : body -> body seq -> vect accels b s = net acceleration on b due to gravitational attraction of the bodies in s

accels accels : body -> body seq -> vect accels b s = net acceleration on b due to gravitational attraction of the bodies in s accels b < b 1 ,...,b n > = accel b b 1 + ... + accel b b n

accels accels : body -> body seq -> vect accels b s = net acceleration on b due to gravitational attraction of the bodies in s accels b < b 1 ,...,b n > = accel b b 1 + ... + accel b b n (vector sum)

accels accels : body -> body seq -> vect accels b s = net acceleration on b due to gravitational attraction of the bodies in s fun accels b s = mapreduce (accel b) add zero s accels b < b 1 ,...,b n > = accel b b 1 + ... + accel b b n (vector sum)