Automatic testing and certification procedure for IGI products in - PowerPoint PPT Presentation

Automatic testing and certification procedure for IGI products in the EMI era and beyond Sara Bertocco INFN Padova on behalf of IGI Release Team Manchester, 10 April 2013 EGI Community Forum 2013

IGI IGI Italian Grid Infrastructure Italian Grid Infrastructure Italian Grid Initiative is a special project of Italian National Institute of Nuclear Physics (INFN). It supports the collaboration between Regional Grids, Compute and Storage Centers, Research Institutions, Centers of Excellence to provide the Italian Grid Infrastructure. The Italian Grid Infrastructure (IGI) services enable and support the access and sharing of data and compute facilities in Italy and worldwide. It guarantees and supports common services for e- research, common middleware releases, general guidelines for policies and standards. EGI Community Forum 2013 - Manchester EGI Community Forum 2013 - Manchester 10 March 2013 10 March 2013 2 2

INFN-IGI Products IGI products included in EMI releases: CREAM CE (Computing Resource Execution and Management - Computing Element) CREAM-UI (CREAM User Interface) LRMS Modules (LSF, Torque, GE, SLURM) CEMON (Computing Element Monitor) Glite-CLUSTER StoRM SE (Storage Resource Manager Storage Element) StoRM UI (StoRM User Interface) WMS (Workload Management System) WMS-UI (WMS User Interface) VOMS VOMS-admin Argus PAP WNoDES IGI products specific of IGI release: • DGAS (DataGrid Accounting System) HLR (Home Location register) GrelC GFAL JAVA APIs Secure Storage System Others EGI Community Forum 2013 - Manchester 10 March 2013 3

EMI Release Cycle EGI Community Forum 2013 - Manchester 10 March 2013 4

EMI Certification Process EGI Community Forum 2013 - Manchester 10 March 2013 5

Goal: Complete testing automation Requirements Repositories Install & config Analyze Git Implement svn Design Puppet Software Development Build&Integration Software testing RobotFramework Jenkins EGI Community Forum 2013 - Manchester 10 March 2013 6

Activities: Testbed Topology CNAF Bologna Testbed Padova Testbed 3 UI : - EMI1 3 VOMS - EMI 2 3 StoRM - gLite 3.2 2 StoRM 1 BDII Top 3 CREAM-CE Torque 2 BDII Site 1 CREAM-CE SGE 8 WNs + 4 WNodes 2 HLR Server 1 ARGUS 3 UI : - EMI1 - EMI 2 5 WMS : - EMI1 1 BDII Site - EMI 2 (SL5) - EMI 2 6 CREAM-CE Torque 2 ARGUS - EMI 3 7 CREAM-CE LSF 1 CREAM-CE SLURM 20 WNs EGI Community Forum 2013 - Manchester 10 March 2013 7

Activities: Testbed Management Redahat Scientific Linux SL5 O.S. Installation DHCP Redahat Scientific Linux SL6 DHCP Centos 5 or 6 PXE Debian PXE Server Server For each physical or virtual machine Middleware Installation Module Puppet Puppet Generic Puppet • ntp Server Server • Egi trustanchor (lcg CA) • Emi-testers-vo Installation Log • emi-prodrepo • emi-rcrepo Log creation Specific - Composed • emi2-argus-deploy • emi-wmstesting-deploy After the puppet modules execution. • emi-creamce-deploy Configuration Log Two log files will be produced EGI Community Forum 2013 - Manchester 10 March 2013 8

Goal: Complete testing automation Requirements Repositories Install & config Analyze Git Implement svn Design Puppet Software Development Build&Integration Software testing RobotFramework Jenkins EGI Community Forum 2013 - Manchester 10 March 2013 9

Continuous Integration We have all the pieces: Source Code Management: GITHUB, SVN CI Server(s): Jenkins Repositories: Nexus Test Automation Framework: RobotFramewrok EGI Community Forum 2013 - Manchester 10 March 2013 10

Real examples EGI Community Forum 2013 - Manchester 10 March 2013

Testing: Robot Framework Robot Framework : test automation framework Main features: can be extended by test libraries Python or Java users can create reusable new keywords from existing ones easy-to-use tabular syntax for creating test cases in a uniform way keyword-driven, data-driven and behavior-driven (BDD) approaches easy-to-read reports and logs in HTML format. tagging to categorize and select test cases to be executed. simple library API for creating customized test libraries. command line interface XML based outputs for integration into existing build infrastructure EGI Community Forum 2013 - Manchester 10 March 2013 12

Testing: Testsuites Management How to write a robot framework integrable test-suite CREAM-BLAH testsuite WMS testsuite VOMS testsuite StoRM testsuite Glite-UI testsuite EGI Community Forum 2013 - Manchester 10 March 2013 13

References Our e-mail IGI Release Team <grid-release@lists.italiangrid.it> Our daily work pages https://wiki.italiangrid.it/twiki/bin/view/IGIRelease/WebHome https://wiki.italiangrid.it/twiki/bin/view/IGIRelease/CertTestBed The testsuites references https://github.com/italiangrid/voms-admin-legacy-testsuite https://github.com/italiangrid/voms-testsuite https://github.com/italiangrid/WMS-Test-Suite https://github.com/italiangrid/storm-testsuite http://nkua-emi.posterous.com/ https://twiki.cern.ch/twiki/bin/view/EMI/CREAMRobotESTests https://twiki.cern.ch/twiki/bin/view/EMI/CREAMStressTests https://wiki.italiangrid.it/twiki/bin/view/IGIRelease/Cream_blah_testsuites https://github.com/italiangrid/cream_blah_testsuites https://github.com/italiangrid/cream_performance_scalability_testshttps: //github.com/danido/EMITestbedTestsuite EGI Community Forum 2013 - Manchester 10 March 2013 14

Thanks ! Questions ? EGI Community Forum 2013 - Manchester 10 March 2013 15