A Minicourse on Multithreaded Programming Charles E Leiserson - PDF document

A Minicourse on Multithreaded Programming Charles E� Leiserson Harald Prok op MIT Lab oratory for Computer Science ��� T ec hnology Square Cam bridge� Massac h usetts ����� f cel�prokop g �l cs �m it �e du July ��� ���� Abstract These notes con tain t w o lectures that teac h m ultithreaded algorithms using a Cilk� lik e ��� � � �� � mo del� These lectures w ere designed for the latter part of the MIT undergraduate class ����� Intr o duction to A lgorithms � The st yle of the lecture notes follo ws that of the textb o ok b y Cormen� Leiserson� and Riv est �� �� but the pseudo co de from that textb o ok has b een �Cilki�ed� to allo w it to describ e m ultithreaded algo� rithms� The �rst lecture teac hes the basics b ehind m ultithreading� including de�ning the measures of w ork and critical�path length� It culminates in the greedy sc heduling theorem due to Graham and Bren t ��� � � �� The second lecture sho ws ho w parallel applications� including matrix m ultiplication and sorting� can b e analyzed using divide� and�conquer recurrences� � Multithreaded programming As m ultipro cessor systems ha v e b ecome increasingly a v ailable� in terest has gro wn in parallel programming� is a programming paradigm in whic h a single Multithr e ade d pr o gr amming program is brok en in to m ultiple thr e ads of con trol whic h in teract to solv e a single problem� These notes pro vide an in tro duction to the analysis of m ultithreaded algorithms� This researc h w as supp orted in part b y the Defense Adv anced Researc h Pro jects Agency �D ARP A� under Gran t F���������������� �

��� Mo del Our mo del of m ultithreaded computation is based on the pro cedure abstraction found in vir� tually an y programming language� As an example� the pro cedure Fib giv es a m ultithreaded algorithm for computing the Fib onacci n um b ers� � Fib � n � � if n � � � then return n � else x � spa wn Fib � n � �� � y � spa wn Fib � n � �� � sync � return � x � y � A spa wn is the parallel analog of an ordinary subroutine call� The k eyw ord spa wn b efore the subroutine call in line � indicates that the subpro cedure Fib � n � �� can execute in parallel with the pro cedure Fib � n � itself� Unlik e an ordinary function call� ho w ev er� where the paren t is not resumed un til after its c hild returns� in the case of a spa wn� the paren t can con tin ue to execute in parallel with the c hild� In this case� the paren t go es on to spa wn Fib � n � ��� In general the paren t can con tin ue to spa wn o� c hildren� pro ducing a high degree of parallelism� A pro cedure cannot safely use the return v alues of the c hildren it has spa wned un til it executes a sync statemen t� If an y of its c hildren ha v e not completed when it executes a sync � the pro cedure susp ends and do es not resume un til all of its c hildren ha v e completed� When all of its c hildren return� execution of the pro cedure resumes at the p oin t immediately follo wing the sync statemen t� In the Fib onacci example� the sync statemen t in line � is required b efore the return statemen t in line � to a v oid the anomaly that w ould o ccur if x and y w ere summed b efore eac h had b een computed� The spa wn and sync k eyw ords sp ecify lo gic al parallelism� not �actual� parallelism� That is� these k eyw ords indicate whic h co de ma y p ossibly execute in parallel� but what ac� tually runs in parallel is determined b y a sche duler � whic h maps the dynamically unfolding computation on to the a v ailable pro cessors� W e can view a m ultithreaded computation in graph�theoretic terms as a dynamically unfolding dag G � � V � E �� as is sho wn in Figure � for Fib � W e de�ne a to b e thr e ad a maximal sequence of instructions not con taining the parallel con trol statemen ts spa wn � sync � and return � Threads mak e up the set V of v ertices of the m ultithreaded computation dag G � Eac h pro cedure execution is a linear c hain of threads� eac h of whic h is connected to its successor in the c hain b y a edge� When a thread u spa wns a thread v � the c ontinuation dag con tains a edge � u� v � � E � as w ell as a con tin uation edge from u to u �s successor sp awn in the pro cedure� When a thread u returns� the dag con tains an edge � u� v �� where v is the thread that immediately follo ws the next sync in the paren t pro cedure� Ev ery computation starts with a single initial thr e ad and �assuming that the computation terminates�� ends � This algorithm is a terrible w a y to compute Fib onacci n um b ers� since it runs in exp onen tial time when logarithmic metho ds are kno wn �� � page ����� but it serv es as a go o d didactic example� �