Open Grid Service Architecture - Data Access & Integration - PowerPoint PPT Presentation

Open Grid Service Architecture - Data Access & Integration (OGSA-DAI) Dave Pearson OGSA-DAI Programme Manager dave.pearson@oracle.com Nov 2003 UK French Grid Workshop

OGSA-DAI Programme Nov 2003 UK French Grid Workshop

OGSA-DAI Motivation • UK e-Science Core Programme – Database Taskforce – Grid data requirements scoping study – Data Access and Integration BoF at GGF4 – DAIS Working Group in Global Grid Forum • Exploit emerging Grid computing concepts (OGSA) • Web Services adoption • DBMS adoption • Meet needs of Grid application developers for UK e-Science community – Grid Pilot projects Nov 2003 UK French Grid Workshop

UK Grid Pilot Projects Research Focus myGrid eScience workbench, Bioinformatics DAME Distributed Aircraft Maintenance Engineering CombiChem Combinatorial (bio-)chemistry RealityGrid Interactive steering of data & compute intensive analysis – chemistry, earth sciences DiscoveryNet Bioinformatics, Environmental & Earth sciences Geodise Computational Fluid Dynamics AstroGrid Virtual Observatory Nov 2003 UK French Grid Workshop

OGSA-DAI Requirements • Transient and persistent data - Data sources & Resources • Maintain complex data models - Information/Knowledge • Record and maintain ‘data about data’ - Metadata • Establish reliability and quality of data - Provenance • Make data more accessible - Publishing and Discovery • Restrict who can read and modify data - Access control • Receive data in a state ready to use - Transformation • Personalise data - Analysis/Interpretation • Quality of Service - Managing data Nov 2003 UK French Grid Workshop

DAI Programme of Work • Two phases – Phase 1 Feb – Sept 2002 complete – Phase 2 Oct 2002 – Jul 2003 – Maintenance activities • Funding for next phase approved - DAIT • Grid data service reference implementations – OGSA compliant – DB2, MySQL, ORACLE, XINDICE • Integration with Globus toolkit • Input to Global Grid Forum on standards recommendations • Collaborative Programme Nov 2003 UK French Grid Workshop

Participating Organisations Edinburgh Parallel Computing Centre North East eScience Centre National eScience Centre North West eScience Centre IBM UK & IBM US Oracle Corporation UK Nov 2003 UK French Grid Workshop

Open Grid Services Architecture • Web services – Standard interface definition mechanisms: multiple protocol bindings, multiple implementations, local/remote transparency – Address discovery & invocation of services • Grid Service extensions – Transient services – State management in a distributed environment • Resource virtualisation • Multiple hosting targets: J2EE, .NET, “C”, … Nov 2003 UK French Grid Workshop

Data Access and Integration Nov 2003 UK French Grid Workshop

OGSA-DAI Positioning - Vision OGSA-DAI Distributed Query OGSA-DAI Basic Services Data Grid Infrastructure – Location, Delivery, Replication… Resource Grid Infrastructure – OGSA… Database, Communication, OS… Technology Nov 2003 UK French Grid Workshop

OGSA-DAI Positioning - Today OGSA-DAI Distributed Query OGSA-DAI Basic Services App Specifics GDSF GDSR GDS Query OGSA (Create Data Retrieve Delivery Format Drivers Update Meta Location Notification Lifetime Delete) Data Database, Communication, OS… Technology Nov 2003 UK French Grid Workshop

OGSA-DAI Behaviours Request to Registry for sources of SOAP/HTTP Registry data about “x” service creation GDSR API interactions Registry responds Request to Factory for access to database with Factory Factory handle GDSF Factory returns handle of Analyst GDS to client Factory creates GridDataService Client queries GDS with SQL, XPath, XQuery etc Query results Grid Data returned XML Service Database OR GDS delivered to consumer (Xindice, MySQL as XML Oracle, DB2) GDS interacts Consumer with database Nov 2003 UK French Grid Workshop

Delivery Patterns Retrieve Update/Insert Pipeline 1. 4. 7. Q Q + U Q1 G1 = P A G A G S S + R S1 I A U/R Q2 + D U 5. 2. Q + D I P S2 G2 = C A Q + D G S A G R S C Q1 + D G1 = P 8. 6. 3. Q P U S1 A A U/R G Q Q2 I S A G D C S R S2 G2 = C Nov 2003 UK French Grid Workshop

Statement Operations • Receives statement • Performs statement against a data resource • Returns results Statement Component and Statement Results Database Connections Data Resource Nov 2003 UK French Grid Workshop

Statement Document <?xml version="1.0" encoding="UTF-8" ?> <GridDataServiceScript> <Header> … Name, versioning, configuration specification </Header> <Body> <Documentation>Update with data delivered with the script</Documentation> <DefineParameter name="table">myimages</DefineParameter> <DefineParameter name="id">id = 2</DefineParameter> <Statement name="xyz" dataResource="MyDataResource"> SELECT * FROM <UseParameter reference="table" /> WHERE <UseParameter reference="id" /> </Statement> <Delivery name="delivery1"> <Mechanism type="bulk" /> <Mode type="full" /> <From>xyz</From> <To>response</To> </Delivery> <Execute name="execute1">xyz</Execute> </Body> Nov 2003 UK French Grid Workshop </GridDataServiceScript>

Statement Architecture Engine/GDS-S Statement Statement Activity Statement Handlers JDBC Statement Handlers XMLDB Statement Handlers JDBC Connection XMLDB Connection XML DB Relational JDBC API XML:DB API DB Nov 2003 UK French Grid Workshop

OGSA-DAI Security • Grid Security Infrastructure Authentication • The role mapper takes credentials and a database name and returns a dbRole • dbRole contains username and password username Credentials Role dbRole Mapper Database Name password Nov 2003 UK French Grid Workshop

OGSA-DAI Product Nov 2003 UK French Grid Workshop

OGSA-DAI Product • Current release 3.0.2 • OGSA-DAI – 1183 downloads as at 27th Oct 2003 – 461 downloads for R3 & R3.0.2 – >350 UK downloads • DQP prototype – 77 downloads since 1st Sept 2003 • Web site – 471 registered users – Transfers to users gigabytes / month • Aug 1.38GB Sep 1.63GB Oct 1.81GB Binary, source and documentation available for download from: www.ogsa-dai.org.uk Nov 2003 UK French Grid Workshop

DAIT Next Phase of Development Nov 2003 UK French Grid Workshop

OGSA-DAI road map 1 • R3.1.0 Jan 04 – Tech. Preview part of R4 • User Group: inaugural meeting Q1 04 • R4.0.0 April 04 – Performance & monitoring – Additional DBMS’s supported – Additional SQL supported – DBMS management operations • archive, restore, bulk load – File access – Client libraries – Installation wizard – User support, courses, training material, performance report Nov 2003 UK French Grid Workshop

OGSA-DAI road map 2 • R5 October 04 – Compliance with DAIS standards proposal – Distributed Relational Query Processing – Improved dependability and security integration – Extended & integrated XML and relational facilities – Distributed transaction participation – Coordinated OGSA-DAI contributor community • R6 April 05 – Integrated with GT3 – New facilities depend on user priorities, context and research – OGSA-DAI components from contributor community • R7 October 05 – Maintainable release for the user community Nov 2003 UK French Grid Workshop