Race Why is parallelism hard? Non-determinism!! Practice Theory - PowerPoint PPT Presentation

Parallel Algorithms: Theory and Practice CS26 S260 – Lecture cture 9* Yan n Gu Race

Why is parallelism “hard”? Non-determinism!! Practice Theory 2

Why is parallelism “hard”? Non-determinism!! • Sc Schedu duli ling ng is is unkno nown wn • Rela lativ ive e orderin ing g for operatio tions ns is is un unknown nown • Hard to d debug ug • Bugs can be non-deterministic! • Bugs can be different if you rerun the code • Referred to as race hazard / condition 3

Race hazard can cause severe consequences • Therac-25 radia iation tion therapy py mach chin ine e — kil ille led 3 3 people le and d serio iously usly in injur ured ed many y more (betwee ween n 1985 a and d 1987). https:// //en.wiki wikiped pedia. a.org/ rg/wiki wiki/Th /Therac rac-25 25 • North America ican n Bla lackout ut of 2003 — le 2003 left 50 m mil illi lion n peopl ple e wit ithout ut power for up to a a w week. . https ps:// //en. en.wiki kiped pedia.org org/w /wiki ki/N /North orthea east st_bl blacko ckout_ t_of_ of_2 003 003 • Race ce bugs s are notorio oriousl usly y difficult icult to discover cover by y co conventional nventional testing! ting! 4

Determinacy Races • Definiti inition: n: a deter ermina minacy cy race occurs when n two lo logi gicall lly para ralle llel l in instru ructi ctions ons acc ccess ss the same me memo mory ry lo loca catio ion n and at le least t one of the in instru tructio tions ns perform orms s a w writ ite. direct_reduce(A, n) { parallel_for (i=0;i<n;i++) sum = sum + 1; return sum; } 5

Determinacy Races • Definiti inition: n: a deter ermina minacy cy race occurs when n two lo logi gicall lly para ralle llel l in instru ructi ctions ons acc ccess ss the same me memo mory ry lo loca catio ion n and at le least t one of the in instru tructio tions ns perform orms s a w writ ite. sum = 0 direct_reduce(A, n) { parallel_for (i=0;i< 2 ;i++) r0 = sum r1 = sum sum = sum + 1; return sum; r0 += 1 r1 += 1 } sum = r0 sum = r1 Return sum 6

Determinacy Races • Definiti inition: n: a deter ermina minacy cy race occurs when n two lo logi gicall lly para ralle llel l in instru ructi ctions ons acc ccess ss the same me memo mory ry lo loca catio ion n and at least le t one of the in instru tructio tions ns perform orms s a w writ ite. sum = 0 direct_reduce(A, n) { parallel_for (i=0;i< 2 ;i++) 1 4 r0 = sum r1 = sum sum = sum + 1; return sum; 2 5 r0 += 1 r1 += 1 } 3 6 sum = r0 sum = r1 return sum 7

Determinacy Races • Definiti inition: n: a deter ermina minacy cy race occurs when n two lo logi gicall lly para ralle llel l in instru ructi ctions ons acc ccess ss the same me memo mory ry lo loca catio ion n and at least le t one of the in instru tructio tions ns perform orms s a w writ ite. sum = 0 direct_reduce(A, n) { parallel_for (i=0;i< 2 ;i++) 1 3 r0 = sum r1 = sum sum = sum + 1; return sum; 2 4 r0 += 1 r1 += 1 } 5 6 sum = r0 sum = r1 return sum 8

Types of Races • Su Suppose ose that in instruct uction ion A and in instruct uction ion B both access s a lo loca catio ion n x, and suppose ppose that t A∥B (A is parallel to B) . A B Race Type Read Read No race Read Write Read race Write Read Read race Write Write Write race • Two se section ions s of code are in indepe epend ndent ent if if they y have e no determi rminac nacy y races s between een them. m. 9

Avoiding races • Itera rations ions of a para rall llel_for el_for lo loop shoul uld d be in indepen ependent dent • Between een two in in_pa paral alle lel tasks, s, the code of the spaw awned ned chil ild shoul uld d be in indepe epend ndent ent of th the code of the parent nt, , in inclu ludi ding ng code execut cuted ed by addi dition ional al spawne wned d or ca call lled chil ildr dren en 10

Benefit of being race-free • Sc Schedu duli ling ng is is stil ill l unkno nown wn • Re Rela lativ ive e ord rderi ring ng for r opera ratio tions ns is is st stil ill l unkno nown wn • However er, , the comput uted ed valu lue e of ea each in instruc uction tion is is determi rminist nistic! ic! This is is is ea easy y to d debug. ug. • Check the correctness of the sequential execution • Check if the parallel execution is the same as the sequential one • Race detection tion: : gi given a D DAG, G, show w all ll th the races • Fals lse e sharing ring: : nast sty y rela lated ed effec fect Struct { • E.g., updating x.a and x.b in parallel is safe char a, b; but can be inefficient } x; 11