New Storage and Cloud Computing Services Proposed by France Grilles Jérôme PANSANEL <jerome.pansanel@iphc.cnrs.fr> Technical Director at France Grilles & Catherine BISCARAT, Cal LOOMIS, Gilles MATHIEU, Geneviève ROMIER
Contents ● Ongoing Projects at France Grilles ● FG-CLOUD ● FG-iRODS 2 J. Pansanel – EGI Conference on Challenges and Solutions for Big Data Processing on Cloud - 2 -
Ongoing projects at France Grilles France Grilles, overseer of French production grids and manager of the relationship with the user communities, aims at increasing the usage of distributed computing infrastructures. To achieve this objective, three projects are ongoing: ● FG-DIRAC: a national DIRAC instance whose objective is to simplify the access to grid and cloud resources for users. ● FG-CLOUD: a project that aims to federate IaaS-Cloud resources in France. ● FG-iRODS: a storage space based on the iRODS software, which provides virtualized storage. 3 J. Pansanel – EGI Conference on Challenges and Solutions for Big Data Processing on Cloud - 3 -
FG-CLOUD: Objectives & Obstacles ● Objectives • Set up a strategy to develop a French Federated IaaS-Cloud infrastructure • Develop the national infrastructure • Build a policy for the resource usage • Define user services ● Obstacles • Coexistence with existing initiatives (bottom-up approach) • France Grilles strategies should be compliant with third-party strategies (sites, guardianship, partners) • Several solutions are currently used for providing Cloud resources • Cloud resources should be transparently accessible • Demonstrate the benefit of a federated Cloud 4 J. Pansanel – EGI Conference on Challenges and Solutions for Big Data Processing on Cloud - 4 -
FG-CLOUD: Technical Group ● Group composition and functioning • Members: site administrators, developers and advanced users • Bimonthly meeting • Guided by a mission statement • Contact: cloud-tech-l@france-grilles.fr ● Ongoing • Development around SlipStream for using OpenNebula • VM-DIRAC configuration tests for working with OpenNebula and SlipStream • Development of Nagios tests ● Objectives • Do a proof of concept of the French federated Cloud with a real scientific use-case (Cloud Challenge) • Ensure that the Cloud is working in a production environment (service redondancy, monitoring, service catalog, ...) • Extend the available resources and user community 5 J. Pansanel – EGI Conference on Challenges and Solutions for Big Data Processing on Cloud - 5 -
FG-CLOUD: Overview of the Sites ● Production sites • OpenStack@CC-IN2P3 (*) • CloudMip@IRIT • Stratuslab@LAL (*) ● Pre-production sites • OpenStack@IPHC (*) • StratusLab@LLR (*) • StratusLab@LUPM • OpenStack@UNIV-LILLE ● Planned site • StratusLab@LPSC (*) ● More information • http://www.france-grilles.fr/6-Cloud 6 J. Pansanel – EGI Conference on Challenges and Solutions for Big Data Processing on Cloud - 6 -
FG-CLOUD: Achievements and Perspectives ● Achievements • Common authentication system • Single marketplace hosted at LAL • VM-DIRAC and SlipStream available for accessing the resources (OpenStack, StratusLab) • Funding call for helping sites joining EGI Federated Cloud (ongoing) • AUPs are defined ● Perspectives • Validate the use of VM-DIRAC and SlipStream with OpenNebula • Development of rOCCI backend for StratusLab • Launch the Cloud Challenge • Deploy a monitoring solution to ensure infrastructure reliability • Provide an expertise for Cloud deployment 7 J. Pansanel – EGI Conference on Challenges and Solutions for Big Data Processing on Cloud - 7 -
FG-iRODS: iRODS in a few Words • Project started in 2006 (based on SRB) • Release under an Open Source license (BSD) • Developed by the DICE group and several collaborators • Rule Engine applies user-defined policies and rules • Integrate a descriptive metadata system to manipulate data iRODS • Data collections manageable over several sites and heterogeneous hardware MS • Logical organization of files is independent of its Disk Disk physical implementation • Enforcement of data consistency and homogeneity http://irods.org/ 8 J. Pansanel – EGI Conference on Challenges and Solutions for Big Data Processing on Cloud - 8 -
FG-iRODS: French NGI Infrastructure • Coordinated by France Grilles • A single production instance: • Federated resources and workforce • Hosting users from any scientific domain 40 TB • Design for small and medium projects • Open to new resource providers • User support and training iCAT 40 TB 20 TB 9 ● Contact: 9 irods-support-l@france-grilles.fr J. Pansanel – EGI Conference on Challenges and Solutions for Big Data Processing on Cloud - 9 -
FG-iRODS: Achievements • Federated set of resources for a total of 100 TB • Real synergy between the administrators • Reliable and highly available storage • Usage policies is published • First training has been performed (Clermont-Ferrand, February 2014) • First users are currently hosted (proteomics and biological data) • Web Interface available • iRODS clients installed on all grid sites supporting the France Grilles VO • iRODS packaging with GSI support (deb, rpm) • VM appliance provided to access the computing Grid and iRODS 10 J. Pansanel – EGI Conference on Challenges and Solutions for Big Data Processing on Cloud - 10 -
FG-iRODS: Perspectives ● Extend the storage pool with new resource providers ● Find new financial resources to ensure the sustainability of the infrastructure ● Deploy a monitoring solution to ensure infrastructure reliability ● Test the S3 plugin ● Welcome more new users ● Share expertise regarding data management and user support with other groups 11 J. Pansanel – EGI Conference on Challenges and Solutions for Big Data Processing on Cloud - 11 -
Thank you for your attention! 12 J. Pansanel – EGI Conference on Challenges and Solutions for Big Data Processing on Cloud - 12 -
Recommend
More recommend
