Leveraging Parallelware in MAESTRO and EPEEC Contributions by Appentra and Enhancements to Parallelware Manuel Arenaz manuel.arenaz@appentra.com PRACE booth #2033 Thursday, 15 November 2018 | Dallas, US http://www.prace-ri.eu/praceatsc18/
Index European Research: H2020 FETHPC programme ● Projects MAESTRO and EPEEC ○ Parallelware Software before MAESTRO & EPEEC ● Benefits and known limitations ○ MAESTRO: Middleware for memory and data-awareness in workflows ● Contributions by Appentra ○ EPEEC: European joint Effort toward a Highly Productive Programming ● Environment for Heterogeneous Exascale Computing Contributions by Appentra ○ Enhancements to Parallelware Software due to Maestro & EPEEC ●
Index European Research: H2020 FETHPC programme ● Projects MAESTRO and EPEEC ○ Parallelware Software before MAESTRO & EPEEC ● Benefits and known limitations ○ MAESTRO: Middleware for memory and data-awareness in workflows ● Contributions by Appentra ○ EPEEC: European joint Effort toward a Highly Productive Programming ● Environment for Heterogeneous Exascale Computing Contributions by Appentra ○ Enhancements to Parallelware Software due to Maestro & EPEEC ●
European Research: H2020 FETHPC Programme Subtopic b) Exascale system software and management, to advance the state of the art in system software and management for node architectures. How can Parallelware tools help to address these R&D Subtopic a) High productivity programming environments for exascale , to simplify challenges? application software development for large- and extreme-scale systems.
Index European Research: H2020 FETHPC programme ● Projects MAESTRO and EPEEC ○ Parallelware Software before MAESTRO & EPEEC ● Benefits and known limitations ○ MAESTRO: Middleware for memory and data-awareness in workflows ● Contributions by Appentra ○ EPEEC: European joint Effort toward a Highly Productive Programming ● Environment for Heterogeneous Exascale Computing Contributions by Appentra ○ Enhancements to Parallelware Software due to Maestro & EPEEC ●
Parallelware Software before MAESTRO & EPEEC Parallelware Technology (libpw) Parallelware front-end Parallelware middle-end Parallelware back-end C OpenMP 4.5 OpenACC 2.0 Semantic Analysis Multi-Threading Engine Offloading GUI Desktop Emerging Technology
Parallelware Software: Benefits Parallelware Technology (libpw) Parallelware front-end Parallelware middle-end Parallelware back-end C OpenMP 4.5 OpenACC 2.0 Semantic Analysis Multi-Threading Engine Offloading Good training tool for parallel programming, particularly for GPUs. ● GUI Desktop Support for multithreading and offloading in OpenMP 4.5. ● Support for offloading in OpenACC. ● Good user messages explaining why and how a loop can be parallelized. ● Management of data scoping of variables (eg. private, shared, reduction) ● Emerging Good code examples for training. ● Technology Multi-platform: Linux, Windows, MacOSX ●
Parallelware Software: Known Limitations Parallelware Technology (libpw) Parallelware front-end Parallelware middle-end Parallelware back-end C OpenMP 4.5 OpenACC 2.0 Semantic Analysis Multi-Threading Engine Offloading Needs enhancements to analyze “real” applications (eg. MPI+X). ● GUI Desktop Lack of support for structs in C. ● Lack of support for C++ and Fortran. ● Lack of support for tasking and SIMD. ● Lack of support for multiple files and procedure calls. ● Emerging More precise support for data-scoping. ● Technology
Index European Research: H2020 FETHPC programme ● Projects MAESTRO and EPEEC ○ Parallelware Software before MAESTRO & EPEEC ● Benefits and known limitations ○ MAESTRO: Middleware for memory and data-awareness in workflows ● Contributions by Appentra ○ EPEEC: European joint Effort toward a Highly Productive Programming ● Environment for Heterogeneous Exascale Computing Contributions by Appentra ○ Enhancements to Parallelware Software due to Maestro & EPEEC ●
Middleware for memory and data-awareness in workflows www.maestro-data.eu Objective: Build a data-aware and memory-aware middleware framework that addresses ubiquitous problems of data movement in complex ● memory hierarchies and at many levels of the HPC software stack. Partners: Forschungszentrum Julich Gmbh ( Juelich , Germany) - Coordinator Commissariat à l'Énergie atomique et aux Énergies alternatives ( CEA , France) Appentra Solutions SL ( Appentra , Spain) Eidgenoessische Technische Hochschule Zuerich ( ETH Zürich , Switzerland) European Centre for Medium-range Weather Forecasts ( ECMWF , United Kingdom) Seagate Systems UK Limited ( Seagate Systems , United Kingdom) Cray Computer Gmbh ( Cray , Switzerland)
Middleware for memory and data-awareness in workflows www.maestro-data.eu The Maestro project has been set up to tackle one of the most important and difficult problems in HPC, ● namely the orchestration of data across multiple levels of the memory and storage hardware as well as the software stack. Although data movement is now recognized as the primary obstacle to performance efficiency, much of the software stack is not well suited to optimizing data movement, and was instead designed in an age where optimizing arithmetic operations was the priority. The Maestro project aims to capture the data- and memory-aware aspects of applications and the ● software stack into a new middleware layer which will perform basic data movement and optimisation on behalf of the application, also making use of modern memory systems. “ The Maestro project will provide a unique opportunity to challenge traditional approaches for handling data objects and data movements in complex HPC applications and workflows, which will be key for efficient exploitation of future exascale level supercomputers. ” Prof. Dirk Pleiter, Coordinator of the Maestro project.
Contributions by Appentra www.maestro-data.eu Appentra will enhance the Parallelware software for data awareness to support the Maestro ● middleware. Appentra brings to the project its long experience in tools for static analysis of HPC codes, which will be ● used to analyze the application and workflow requirements of the Maestro middleware, and to co-design the Maestro middleware . Appentra brings to the project its long experience in tools for static analysis of HPC codes, which will be ● used to develop new components of the Maestro middleware concerned with data access and dataflow as well as data-aware execution and orchestration . Appentra will contribute to the validation and demonstration of the Maestro middleware. ● Appentra leads the coordination of the dissemination activities in Maestro. ●
Contributions by Appentra www.maestro-data.eu Parallelware Technology (libpw) Parallelware front-end Parallelware middle-end Parallelware back-end OpenMP 4.5 C OpenACC 2.0 Semantic Analysis Multi-Threading Engine Offloading GUI Desktop The Parallelware middle-end will be enhanced by adding the new source code static analysis capabilities needed by the Maestro data orchestration middleware. Emerging Technology
Contributions by Appentra www.maestro-data.eu Parallelware Technology (libpw) Parallelware front-end Parallelware middle-end Parallelware back-end OpenMP 4.5 C OpenACC 2.0 Semantic Analysis Multi-Threading Engine Offloading 1. Preparation of the Parallelware software to expose the information available in GUI Desktop the middle-end : Hidden in Parallelware Trainer as the amount of information would be overwhelming. ○ 2. Two new ways to expose the information of Parallelware middle-end: Parallelware Analyzer , new command-line tool. ○ Extension to libpw , new API for third-party tools to access to Parallelware capabilities. ○ Emerging Technology
Contributions by Appentra www.maestro-data.eu Parallelware Technology (libpw) Parallelware front-end Parallelware middle-end Parallelware back-end OpenMP 4.5 C OpenACC 2.0 Semantic Analysis Multi-Threading Engine Offloading on going 3. First release of Parallelware Analyzer version BETA . Command GUI Desktop Line Tool a. Proposed first set of analyses: --datascoping, --functions, --datalayout, --code and --overview b. Enhancements to the Parallelware middle-end under development: i. Tracking of scalars across multiple files and multiple procedures ii. Tracking of fields of structs Emerging Emerging Technology Technology
Index European Research: H2020 FETHPC programme ● Projects MAESTRO and EPEEC ○ Parallelware Software before MAESTRO & EPEEC ● Benefits and known limitations ○ MAESTRO: Middleware for memory and data-awareness in workflows ● Contributions by Appentra ○ EPEEC: European joint Effort toward a Highly Productive Programming ● Environment for Heterogeneous Exascale Computing Contributions by Appentra ○ Enhancements to Parallelware Software due to Maestro & EPEEC ●
Recommend
More recommend
