A Mediation Framework for Transparent Access to largely distributed data sources Christine Collet Christine.Collet@imag.fr Institut National Polytechnique Grenoble LSR assistant director Database group leader
http://www-lsr.imag.fr/mediagrid Project's Partners LaMI (Lab. des Méthodes Informatiques) Univ. Evry- Val d'Essonne, France LSR (Lab. Logiciels Systèmes Réseaux), IMAG Grenoble, France PRiSM (Lab. Parallélisme, Réseaux, Systèmes, Modélisation), Univ. Versailles St Quentin, France 2
MEDIAGRID objective Contribute to the definition of an open mediation framework for the Grid � « framework » means a reusable design of a mediation system expressed as a set of abstract classes (or components) and the way their instances collaborate « open » refers to the construction of � mediation systems out of heterogeneous (hardware, software and network) elements 3
Mediation system APPLICATION USER Mediation schema MEDIATOR MQn MQ2 M e d i a t i MEDIATOR o n l e MQ1 v e l WRAPPER WRAPPER WRAPPER WRAPPER Exported schema 4
Several aspects to consider Structured : DISCO (objet), Structured : DISCO (objet), GAV : TSIMMIS, Garlic, MiX, GAV : TSIMMIS, Garlic, MiX, Design LeSelect (relationnel) Design LeSelect (relationnel) Single mediator : LeSelect, Hermes, MOMIS, Xyleme, Single mediator : LeSelect, Hermes, MOMIS, Xyleme, Information Manifold, … YAT, … Communication Information Manifold, … YAT, … Communication Semi-structured : TSIMMIS (OEM), Semi-structured : TSIMMIS (OEM), Tukwila , MIX , Execution Tukwila , MIX , Execution Hierarchie of mediators : TSIMMIS, DISCO, … LAV : Information Manifold, SIMS, Hierarchie of mediators : TSIMMIS, DISCO, … LAV : Information Manifold, SIMS, YAT, MOMIS (XML) YAT, MOMIS (XML) Tukwila, PICSEL, Agora, DWQ,… Tukwila, PICSEL, Agora, DWQ,… Association Association Systems more and more distributed Systems more and more distributed Syntax (technics, interfaces) Syntax (technics, interfaces) Several protocols (SOAP, CORBA, RMI) Several protocols (SOAP, CORBA, RMI) Logic (semantics, schemas) Logic (semantics, schemas) Several transparency levels Several transparency levels classical : TSIMMIS, Garlic, MiX, classical : TSIMMIS, Garlic, MiX, Hermes, Xyleme, Inf. Manifold Hermes, Xyleme, Inf. Manifold Yes : IRO-DB, DWQ, IGD, … Yes : IRO-DB, DWQ, IGD, … Thin wrapper : Information Manifold, Thin wrapper : Information Manifold, YAT, … YAT, … SIMS, … SIMS, … No : Information Manifold, SIMS, No : Information Manifold, SIMS, ontology-based : SIMS, Ontobroker, MOMIS, ontology-based : SIMS, Ontobroker, MOMIS, Tukwila, PICSEL, Agora, DWQ, Tukwila, PICSEL, Agora, DWQ, Thick wrapper : TSIMMIS, Garlic, … Thick wrapper : TSIMMIS, Garlic, … OBSERVER, DWQ, … OBSERVER, DWQ, … TSIMMIS, Garlic, MiX, TSIMMIS, Garlic, MiX, Hermes, MOMIS, Xyleme, YAT, … Hermes, MOMIS, Xyleme, YAT, … 5
Revisiting … Huge amount of knowledge to maintain (sources � descriptions, schemas, semantic relations between schemas) Growing complexity with respect to the number, the � types and capacities of sources High dynamicity: data sources evolution, new � sources being of interest, remove sources Long running queries and continuous queries: may � need to change the execution plan, to produce partial results, to materialize results Interaction: Users and applications want to control � the query processing 6
User level Response Metadata XQuery XQuery XQuery For Eukaryotes organisms For Eukaryotes organisms For Where Where Entirely sequenced Where Entirely sequenced Return Expression matrix Return Expression matrix Return XML schema Mediation level Mediation level Mediation schema GAV Mediation queries MS=Q(S 1 ,S 2 ,S 3 ) Rewriting LAV Mediation queries S 1 = Q(MS), S 2 =Q(MS) Mediation Query in terms of Query in terms of Queries Semantic exported schemas exported schemas correspondences generator Evaluator Statistics Exported schema Schéma exporté (XML schema) (XML schéma) Sub-query Sub - query 1 Sub-query Sub - query n Capabilities Intermediary result 1 Intermediary result n Source level Wrapper Wrapper S 1 S n MEDIAGRID big picture 7
Generation of mediation queries � Given an entity R Mediation Schema MEDATION within the mediation R(#K, A, B, C) schema and exported schemas Q: R1 R2 R3 � How to generate SOURCES mediation queries R1(#K, A) R3(#B, C, D) computing data of R R2(#A, B) R4(#D, E) from data of sources Source S1 Source S2 8
Generation of mediation queries � Extract relevant portions of a source (mapping schemas) for a given mediation schema. -> mapping for R : (K, A) of S1/R1, (A, B) of S1/R2 and (B,C) of S2/R3 � Find the candidate operations between mapping schemas using rules such as If two mapping schemas have an overlapping sub-tree then the join operation is candidate between the two mapping schemas. � Generate mediation queries Ex: R1(K, A) R2(A, B) R3(B,C) 9
Adaptive and interactive query evaluator Adaptive : it is able to adapt itself to the execution environment (in case a query takes a long time to produce results, data arrive after a too long time, too much data has already been processed …) Interactive : it allows user control. Users specify their interests or constraints on processing queries. 10
QBF architecture Context IContextMgr Manager Rule IRuleMgr Manager Query Plan IPlanMgr Manager Manager IQueryMgr Monitor IMonitor Buffer Legends Manager provided interface IBufferMgr 11 required interface
Adaptive query evaluation Monitor and RuleManager Classes have been defined to monitor: arrival data rate, number of data processed, and execution time. op1’ op1 op1 retransform buffer S Buffer T S op2 op2 op2’ S T R R T R [Query Scrambling technic] when timeout when timeout if not began if not transformable do retransform do buffer 12
Interactive query evaluation � User interest on � number of results, execution time, data preference, type of partial result, … � Refinement : operations � For modifying context parameters, requesting partial results, preference, halting operators, adding new operators, … � Managed as � MonitoringProperty instances, rules, control operators (e.g. Counter, Partial, Buffer…) � With techniques : monitoring, active rules, construction of partial results, … 13
Contribution to Data GRID � One of the key challenge: the provision of system and data management services in the large scale, open and highly dynamic environment of a GRID � MEDIAGRID contributions: frameworks for data integration, mediation queries generation, adaptive and interactive query evaluation, wrapper / mediator generation, rules, workflows, metadata … � More generally, DBMS components as middelware open services for data management: caching, persistency, replication, transactions, confidentiality, security, querying, workflows, composition, etc. 14
Possible links in UK � The OGSA-DAI project http://www.ogsa-dai.org.uk/ � concerned with constructing middleware to assist with access and integration of data from separate data sources via the grid. � conceived by the UK Database Task Force and is working closely with the Global Grid Forum DAIS-WG and the Globus team. The Open Grid Service Architecture includes a Grid Distributed Query Service (GDQS) and a Grid Query Evaluation Service (GQES). � distribution and implicit parallelism in query processing 15
CoreGrid & ObjectWeb � The ObjectWeb consortium can provide valuable technology and exploitation path for CoreGrid � check out http://www.objectweb.org � ObjectWeb focus is the development of open source distributed middleware � current emphasis is on Web application servers � plans are for the development of highly configurable component- based middleware (with multiple application domains in mind) � distributed system technology and data (base) management technology in ObjectWeb can be useful for Grids esp. when it comes to merge Grid infrastructure and Web services infrastructure 16
Recommend
More recommend
