Efficient Observations Forecast for the Worlds Biggest Eye Using - PowerPoint PPT Presentation

Efficient Observations Forecast for the World’s Biggest Eye Using DGX-1 Damien Gratadour 1 and Hatem Ltaief 2 1 LESIA, Observatoire de Paris and Universit´ e Paris Diderot, France 2 Extreme Computing Research Center, KAUST, Saudi Arabia NVIDIA GTC at San Jose, CA May 8-11, 2017 D. Gratadour & H. Ltaief MOAO Using DGX-1 1 / 48

Outline The European Extremely Large Telescope 1 Leveraging Task-based Programming Model 2 Parallel Implementation 3 Performance Results 4 Conclusion 5 D. Gratadour & H. Ltaief MOAO Using DGX-1 2 / 48

Acknowledgments Students/Collaborators Extreme Computing Research Center @ KAUST A. Charara and D. Keyes L’Observatoire de Paris, LESIA R. Dembet, N. Doucet, E. Gendron, D. Gratadour, C. Morel, A. Sevin and F. Vidal Innovative Computing Laboratory @ UTK PLASMA/MAGMA/PaRSEC Teams INRIA/INP Bordeaux, France Runtime/HiePACS Teams D. Gratadour & H. Ltaief MOAO Using DGX-1 3 / 48

Acknowledgments Support/Funding KAUST IT Research Computing support NVIDIA hardware donations ( can we get more? ∗ ) Funded partially by the French National Center for Scientific Research (CNRS, 2016) Funded partially by the European Commission (Horizon 2020 program, FET-HPC grant# 671662) ∗ For free D. Gratadour & H. Ltaief MOAO Using DGX-1 4 / 48

E-ELT Outline The European Extremely Large Telescope 1 Leveraging Task-based Programming Model 2 Parallel Implementation 3 Performance Results 4 Conclusion 5 D. Gratadour & H. Ltaief MOAO Using DGX-1 5 / 48

E-ELT The effect of atmospheric turbulence The sun observed with a compact camera Disturbs the trajectory of light rays (wavefront perturbations) Reduces astronomical images quality D. Gratadour & H. Ltaief MOAO Using DGX-1 6 / 48

E-ELT Adaptive optics (AO) AO: technique used compensate in real-time the wavefront perturbations providing a significant improvement in resolution The moon observed with a 8m telescope (left: no AO, right: with AO) D. Gratadour & H. Ltaief MOAO Using DGX-1 7 / 48

E-ELT How AO works D. Gratadour & H. Ltaief MOAO Using DGX-1 8 / 48

E-ELT The Top 10 (present and future) Ground-based Telescopes Rank Name Location Diameter Cost Year 10 Large Synoptic Survey Telescope (LSST) Chile 8.4m 450 million 2014 9 South African Large Telescope (SALT) South Africa 9.2m 36 million 2005 8 Keck USA 10m 100 million 1996 7 Gran Telescopio Canarias (GTC) Spain 10.4m 130 million 2009 6 Aricebo Observatory Puerto Rico 305m 9.3 million 1963 D. Gratadour & H. Ltaief MOAO Using DGX-1 9 / 48

E-ELT The Top 10 (present and future) Ground-based Telescopes Rank Name Location Diameter Cost Year 10 Large Synoptic Survey Telescope (LSST) Chile 8.4m 450 million 2014 9 South African Large Telescope (SALT) South Africa 9.2m 36 million 2005 8 Keck USA 10m 100 million 1996 7 Gran Telescopio Canarias (GTC) Spain 10.4m 130 million 2009 6 Aricebo Observatory Puerto Rico 305m 9.3 million 1963 5 Atacama Large Millimeter Array (ALMA) Chile 12m 1.4 billion 2013 4 Giant Magellan Telescope (GMT) Chile 24.5m 2.2 billion 2024 3 Thirty Meter Telescope (TMT) USA 30m 1.4 billion 2030 2 Square Kilometer Array (SKA) Australia 90m 2 billion 2020 1 European Extremely Large Telescope (E-ELT) Chile 39m 1.3 billion 2024 D. Gratadour & H. Ltaief MOAO Using DGX-1 9 / 48

E-ELT The Top 10 (present and future) Ground-based Telescopes Rank Name Location Diameter Cost Year 10 Large Synoptic Survey Telescope (LSST) Chile 8.4m 450 million 2014 9 South African Large Telescope (SALT) South Africa 9.2m 36 million 2005 8 Keck USA 10m 100 million 1996 7 Gran Telescopio Canarias (GTC) Spain 10.4m 130 million 2009 6 Aricebo Observatory Puerto Rico 305m 9.3 million 1963 5 Atacama Large Millimeter Array (ALMA) Chile 12m 1.4 billion 2013 4 Giant Magellan Telescope (GMT) Chile 24.5m 2.2 billion 2024 3 Thirty Meter Telescope (TMT) USA 30m 1.4 billion 2030 2 Square Kilometer Array (SKA) Australia 90m 2 billion 2020 1 European Extremely Large Telescope (E-ELT) Chile 39m 1.3 billion 2024 Consortium: multiple nation initiatives Src: http://www.space.com/14075-10-biggest-telescopes-earth-comparison.html D. Gratadour & H. Ltaief MOAO Using DGX-1 9 / 48

E-ELT The World’s Biggest Eye on The Sky Credits: ESO (http://www.eso.org/public/teles-instr/e-elt/) D. Gratadour & H. Ltaief MOAO Using DGX-1 10 / 48

E-ELT The World’s Biggest Eye on The Sky Credits: ESO (http://www.eso.org/public/teles-instr/e-elt/) The largest optical/near-infrared telescope in the world. It will weigh about 2700 tons with a main mirror diameter of 39m. Location: Chile, South America. D. Gratadour & H. Ltaief MOAO Using DGX-1 11 / 48

E-ELT Multi-object Adaptive Optics (MOAO) Probably the most challenging embedded instrument in the E-ELT. Observe the most distant galaxies in parallel to understand their evolution with a high multiplex Capable of exploiting a Field of View (FoV) of 7 to 10 arcminutes (1/4 of the full moon) with a resolution of few tens of milli-arcsec (1/20,000 of the full moon) Use multiple guide stars (tomographic measurement of the turbulence) and multiple deformable mirrors (direction specific compensation). Extremely compute intensive at full scale. D. Gratadour & H. Ltaief MOAO Using DGX-1 12 / 48

E-ELT Multi-object Adaptive Optics (MOAO) Credits: ESO D. Gratadour & H. Ltaief MOAO Using DGX-1 13 / 48

E-ELT Multi-object Adaptive Optics (MOAO) Creating artificial guide stars. Credits : A. Reeves, Durham University D. Gratadour & H. Ltaief MOAO Using DGX-1 14 / 48

E-ELT Multi-object Adaptive Optics (MOAO) Probably the most challenging embedded instrument in the E-ELT. Extremely compute intensive at full scale. Today: need to simulate the system to provide efficient observations forecast (evaluate the possible science return, perform design studies). Tomorrow (2024): need to drive the system in real-time for routine operations Core component of the system: the real-time controller which provides commands to the deformable mirrors actuators from the measurements of multiple wavefront sensors. D. Gratadour & H. Ltaief MOAO Using DGX-1 15 / 48

E-ELT AO real-time controller Heterogeneous HPC facility D. Gratadour & H. Ltaief MOAO Using DGX-1 16 / 48

E-ELT AO real-time controller Heterogeneous HPC facility Real-time box: stay tuned for J. Bernard presentation later in this session about our implementation on DGX-1 D. Gratadour & H. Ltaief MOAO Using DGX-1 17 / 48

E-ELT AO real-time controller Heterogeneous HPC facility Here we concentrate on the supervisor sub-system Critical component for operations and observations forecast D. Gratadour & H. Ltaief MOAO Using DGX-1 18 / 48

E-ELT AO supervisor sub-system Cost function optimization for parameters identification ( Learn stage) Linear algebra for reconstructor matrix computation ( Apply stage) D. Gratadour & H. Ltaief MOAO Using DGX-1 19 / 48

E-ELT Learn process Fitting a covariance matrix on a model including system and turbulence parameters, using a score function Levenberg-Marquardt algorithm for function optimization Example of fitted turbulence profile D. Gratadour & H. Ltaief MOAO Using DGX-1 20 / 48

E-ELT Learn process Multi-GPU process, including matrix generation and LM fit Time to solution on DGX-1 for a matrix size of 86k and 40 turbulence layers:240s (4 minutes) Weak and strong scaling for the learn process D. Gratadour & H. Ltaief MOAO Using DGX-1 21 / 48

E-ELT Apply process: tomographic reconstructor Compute the tomographic reconstructor matrix using covariance matrix between direction specific truth sensor and other sensors and the inverse of measurements covariance matrix R ′ = C tm · C − 1 mm Factorize and Solve for R ′ with C mm , a 100k x 100k matrix, is extremely compute intensive At the core of system operations (soft real-time, should be achieved in seconds to update the real-time box) Also a critical component for the numerical simulation of the system behavior (observation forecast for today’s design studies) D. Gratadour & H. Ltaief MOAO Using DGX-1 22 / 48

E-ELT AO simulation pipeline: observations forecast Goal is to produce image quality maps over the instrument’s full FoV depending on turbulence conditions evolution over the night D. Gratadour & H. Ltaief MOAO Using DGX-1 23 / 48

E-ELT AO simulation pipeline: observations forecast From system parameters and turbulence evolution timeline to image quality at the output of the system D. Gratadour & H. Ltaief MOAO Using DGX-1 24 / 48

Task Model Outline The European Extremely Large Telescope 1 Leveraging Task-based Programming Model 2 Parallel Implementation 3 Performance Results 4 Conclusion 5 D. Gratadour & H. Ltaief MOAO Using DGX-1 25 / 48

Task Model One of The Possible Solutions for Exascale Computing Fine-granularity computation Asynchronous execution High concurrency Runtime system for separation of concerns Productivity with abstraction Performance/debugging tools Popular: available from OpenMP 3.0 D. Gratadour & H. Ltaief MOAO Using DGX-1 26 / 48