Employing Task Parallelism to Facilitate Dynamic Comparison of - PowerPoint PPT Presentation

Employing Task Parallelism to Facilitate Dynamic Comparison of Model Output Ha nna h Aize nma n (CUNY Gra d ua te Ce nte r, NOAA-CRE ST a t CCNY) Me nto rs: Da ve Bro wn, Ma ry Ha le y (NCAR-CISL / VE T S) Co ntrib uto rs: She ri Mic ke lso n (ANL ), Susa n Ba te s, Go kha n Da na b a so g lu (NCAR-CGD)

OMWG POP Diagnostics

Swift POP Diagnostics Pa ra lle lize s e xe c utio n o f NCL sc ripts • 3X spe e d up o ve r No n-Swift ve rsio n •

Only supported on DASG machines

Even on DASG…

NCAR’s mission is “to support, enhance, and extend the capabilities of the university community and the broader scientific community, nationally and internationally” - http://ncar.ucar.edu/about-ncar

Service Oriented Architecture

Why SOA? Se rvic e s sta y inde pe nde nt • Dia g no stic s c a n b e a dde d inde pe nde nt o f o the r se rvic e s o Ne w se rvic e s c a n use o ld o ne s witho ut c ha ng ing the o lds o ne s o Multiple c lie nts c a n ta lk to the sa me se rve r • CL I , mo b ile , de skto p, … o Ca n ke e p o ld UI up while ro lling o ut ne w o ne o Clie nts a nd se rvic e s do no t ne e d to b e o n the sa me • ma c hine Slo w se rvic e s (the dia g no stic s) do no t inte rfe re with • fa st o ne s (da ta , o b se rva tio ns)

RESTful SOA

Start a run

Get the results

URL->File

Why pydiag?

Automate running the diagnostics T urn use r de fine d se tting s into inputs • c a se / run, o b se rva tio ns, ye a rs, a nd plo ts o I nputs a re pytho n dic tio na rie s o Ge ne ra te Swift c o nfig ura tio n file s • site s.xml c ha ng e s with e ve ry ne w o utput dire c to ry o site s.xml a nd tc .da ta a re ma c hine de pe nde nt o Co nfig s a re dic tio na rie s a nd lists o f dic ts o F ile s g e ne ra te xml, tsv, c dm, a nd c o nfig ura tio n • I so la te de pe nde nc ie s • T urn it into a n impo rta b le lib ra ry •

CSH->Python

Deployment Re q uire s a ny lib ra rie s use d b y the dia g no stic s • Se lf-c o nta ine d e nviro nme nt • I so la te d thro ug h use o f virtua le nv o All the pa c ka g e s a re in PyPi o Only o the r de pe nde nc ie s o lib e ve nt a nd a c c e ss to the da ta se rve rs (g la de , hpss) • Wo rks b e tte r o n c luste r o Ma inte na nc e a nd e xte nsib ility: • Mo dula r a rc hite c ture o Unit, inte g ra tio n, a nd func tio na l re g re ssio n te sts o Do c ume nta tio n c a n b e c o mpile d into we b pa g e s, b o o ks, e tc o pydia g , whic h is sta nd-a lo ne , is c o mpa tib le with • Pytho n2.4 thro ug h Pytho n2.7

Server Side mik.ucar.edu Use any of the documented/sample urls

Client Side

Acknowledgments Da ve Bro wn a nd Ma ry Ha ly (NCAR/ CI SL ) • She ri Mic ke lso n (ANL ) • Susa n Ba te s a nd Go kha n Da na b a so g lu • (NCAR/ CGD) Pa ul Go o dma n a nd Na tha n Wilhe lmi (NCAR-CI SL ) • We i Hua ng a nd Ric k Bro wnrig g (NCAR/ CI SL ) •