Experimental Federated Geoprocessing Cloud Services for Earth - PowerPoint PPT Presentation

Experimental Federated Geoprocessing Cloud Services for Earth Science Community H. Astsatryan, A. Hayrapetyan, W. Narsisian Institute for Informatics and Automation Problems, National Academy of Sciences of the Republic of Armenia

Institute for Informatics and Automation Problems of the National Academy of Sciences of Armenia (IIAP) ü Founded in 1957 as the only state supported structure in Armenia for software, hardware and brainware technologies ü Operates as a State Non- Profit Organization ü Staff >150 ü More at http://iap.sci.am IIAP in a national level coordinates the e-infrastructures of the Academic Scientific Research Computer Network of Armenia (ASNET-AM National Research and Education network) a and the Armenian National Grid Initiative (ArmNGI). ü ASNET-AM provides network connectivity and services to more than 60 organizations in 6 cities of Armenia ü ArmNGI is a common resource (more than 500 cores) of the scientific and academic community and represents the advanced infrastructure for the scientific research, the application of new technologies EGI Conference on Solutions and Challenges for Big Data Processing, 23-26 September 2014, Amsterdam, The Netherlands

Earth Science Community in Armenia Due to its intensive data processing, the multidisciplinary Earth Science communities in Armenia needs large-scale resources for different calculations, and an efficient storage to keep their data and meta-data, such as Five observational instruments are installed in the Byurakan Astrophysical Observatory on the picturesque southern slope of the mountain Aragatz. At the observatory the larger ones being 2.6 m Cassegrain telescope and 1m Schmidt telescope. Implementation of a mesoscale Weather Research and Forecasting model for the territory of Armenia with fine spatial resolution, which is a big and complex application that requires lot of different input and output data National spatial data infrastructure is a common framework enabling the efficient aquisition, distribution and use of geospatial information. EGI Conference on Solutions and Challenges for Big Data Processing, 23-26 September 2014, Amsterdam, The Netherlands

Earth Science Community in Armenia: Needs and Challenges Challenges ü To implement the the international standards (such as Open Geospatial Consortium) that cater that allow interoperability between communities that previously were not able to easily communicate. ü To develop cloud computing based workflows in geospatial sciences consist of data storage and processing, as well as simulation and modeling. ü Effective Exploitation of HPC resources for earth Science community. Needs ü To support big geospatial data analysis by building a cloud-enabled scientific workflow platform, such as processing of high-resolution time series satellite images typically requires a large amount of computational resources and time. ü To support Earth science data tackling by providing Cloud Storage: ü Single Sign-On for the community enabling to gain access to multiple resources by authenticating only once EGI Conference on Solutions and Challenges for Big Data Processing, 23-26 September 2014, Amsterdam, The Netherlands

Experimental Federated Geoprocessing Cloud Services (a) Processing of satellite imagery is a science of bringing measurements, units to the satellite image pixel, which are commonly found in terms of indices (vegetation indices /VI/ especially), some more physically based information can be derived and units are then starting to appear. The suggested environment used for the calculation of a) Single vegetation among 12 vegetation indices. EGI Conference on Solutions and Challenges for Big Data Processing, 23-26 September 2014, Amsterdam, The Netherlands

Experimental Federated Geoprocessing Cloud Services (a) a) Single vegetation from12 vegetation indices. EGI Conference on Solutions and Challenges for Big Data Processing, 23-26 September 2014, Amsterdam, The Netherlands

Experimental Federated Geoprocessing Cloud Services (b) b) Workflow as a combination of vegetation indices. EGI Conference on Solutions and Challenges for Big Data Processing, 23-26 September 2014, Amsterdam, The Netherlands

Experimental Federated Geoprocessing Cloud Services (c) c) Times Series analyzes - In the front end, the user may choose an appropriate scene from the available local data repository by indicating the season and the required time period ! EGI Conference on Solutions and Challenges for Big Data Processing, 23-26 September 2014, Amsterdam, The Netherlands

Experimental Federated Geoprocessing Cloud Services Geoprocessing Services Combination of Single index Time series analyzes indices EGI Conference on Solutions and Challenges for Big Data Processing, 23-26 September 2014, Amsterdam, The Netherlands

Experimental Federated Geoprocessing Cloud Services Geoprocessing Services Combination of Single index Time series analyzes indices Local Cloud The Web Processing Service standard is used to handle the requests from and the responses to the services. The service trader module receives the requested information from the website and decides the target platform (local, cloud) EGI Conference on Solutions and Challenges for Big Data Processing, 23-26 September 2014, Amsterdam, The Netherlands

Cloud Storage Implementations and Studies A Federated Cloud Platform in the BSEC Region based on OpenNebula Zone (oZone), which is a group of physical hosts under the control a single OpenNebula instance H. Astsatryan, A. Hayrapetyan, W. Narsisian, V. Sahakyan, Yu. Shoukourian, G. Neagu, and A. Stanciu, Environmental Science Federated Cloud Platform in the BSEC Region, International Journal of Scientific & Engineering Research, vol. 1, no.1, pp. 1130-1133, January 2014, ISSN 2229-5518. Federated and interoperable OpenStack clouds Synnefo: A Complete Cloud Stack over Ganeti EGI Conference on Solutions and Challenges for Big Data Processing, 23-26 September 2014, Amsterdam, The Netherlands

Experimental Federated Geoprocessing Cloud Services Geoprocessing Services Combination of Single index Time series analyzes indices Local Cloud GRASS GIS An effective algorithm for parallelization within the single multicore node has been developed in order to speed up image processing. H. Astsatryan, W. Narsisian, A. Hayrapetyan, A. Saribekyan, Sh. Asmaryan, V. Muradyan, Y. Guigoz, G. Giuliani, N. Ray, An Interoperable Web Portal for Parallel Geoprocessing of Satellite Image Indices, Springer Earth Science Informatics, 2014, DOI: 10.1007/s12145-014-0165-3. EGI Conference on Solutions and Challenges for Big Data Processing, 23-26 September 2014, Amsterdam, The Netherlands

Cloud Storage Service A cloud storage, based on Synnefo/Pithos and uses a CEPH OSD storage system. ü Omit a big download/upload time, because it is in the same network. ü Data security with an encryption mechanism. ü Prevent data loose, multiple copy for the data. ü Data sharing. EGI Conference on Solutions and Challenges for Big Data Processing, 23-26 September 2014, Amsterdam, The Netherlands

Single sign on for Cloud Storage & Geoprocessing Services Centralized place to have an access to all services. The unified service keeps the identity information of the community members in one centralized place and provides all these different components to the Earth Science Community. Cloud storage Single Sign in Geoprocessing services Shibboleth is among the world's most widely deployed federated identity solutions, connecting users to applications both within and between organizations. EGI Conference on Solutions and Challenges for Big Data Processing, 23-26 September 2014, Amsterdam, The Netherlands