From GeoSpatial to BioSpatial: Managing 3D Structure Data Xavier R. Lopez Director, Location Services Oracle Corp.
Overview � Market & Technology Trends � Spatial Database Technology � GeoSpatial DBMS in GeoSciences � Life Sciences Data Management Challenges � BioSpatial DBMS in Life Sciences
Spatial data becoming ubiquitous � Location Aware and Enabled Infrastructure – Defense, Logistics, Mobile devices � Internet Portals: MapQuest, Yahoo, MapPoint.NET � Automobiles: by 2006, 80% of new cars will have some telematics navigation access (eyeforauto 2001) � Structure Databases: Proteomics, Materials Science
Spatial Analysis Revealing patterns, relationships & trends Manage Location Client Name Usage AUSTRIA **Hallein Municipality Local authority resources AUSTRIA **Lu desch Local Gov ernment AUSTRIA ARG Verrmessu ng, Do rnbirn Surv ey and mappin g AUSTRIA ILF-Dornbirn -8 AUSTRIA ILF-Innsbrueck - 2 Discover AUSTRIA ILF-Prague - 2 AUSTRIA ILF-Vienna - 2 AUSTRIA ILF-Villah - 1 AUSTRIA Inge nieurgemeinschaft Laesser-Fezlmayr (ILF), Engineering company demographic AUSTRIA Lochau Municipality, Vorarlberg Local gov ernment AUSTRIA Manahl, Feldkirch Engineering company AUSTRIA Vorarlberg Erdgas, Dornbirn Gas distribution trends BOSNIA City of Zage b(CV) Local gov ernment BOSNIA Computech (CV) Reseller BRAZIL Systenge Reseller CANADA City of Edmonton Local gov ernment CANADA City of Ludu c Local gov ernment CANADA District of Oak Bay Local gov ernment CANADA Energy & Mines (Ottawa) CANADA Energy & Mines (Quebec) CANADA Geo power T echnolo gies, Inc. Reseller CANADA H.H. Pillar Corp. CANADA Univ ersity of Toronto Education CHINA Beihai Urban Construction CHINA Beijing Urban Archive Local gov ernment FINLAND Pohjois-Satakun nan paikkatietopalv elu OY GIS systems house FINLAND Tampere muncipality (PCX 100 USER LICENCE) Local gov ernment FRANCE Cabinet Dulac Surv ey and mappin g FRANCE District Bayonne - Anglet - Biarritz Local gov ernment consortium FRANCE EPA Cergy-Pontoise New town dev elopment FRANCE France Telecom Telecommunic. company FRANCE Gaz de France Gas distribtuion FRANCE Institut Geographique National (IGN) National mapping agency FRANCE ITMI Software dev eloper/integrator FRANCE Municipality of Dijon Local gov ernment Locate a FRANCE Nancy District Local gov ernment FRANCE School of IGN IGN's training school FRANCE Univ ersity of Caen Educationa l new facility Reveal travel patterns
Overcoming Application “ Stovepipes ” � Specialty GIS servers GIS Solution GIS Solution Enterpri Enterprise Solution e Solution Data isolation – High systems admin GIS GIS GIS GIS – Database Database Database Database Applicatio ications ns Applicatio ications ns Applicatio ications ns Applicatio ications ns and management costs GIS GIS Enterprise Enterprise Scalability problems – High training costs – Spatial Spa Spatial Spa Complex support RDBMS RDBMS RDBMS RDBMS – Data Da ta Data Da ta problems � Information not aligned with Business Processes � Applications can ’ t • Billing • Presence leverage brute force of • Personalization large servers
Life Sciences: Drug Discovery � The Process Public Databases Local Databases Industrial Research Lab. Local Copies Partner or Collaborator Private/Service Databases
Many Different Kinds Data Genomics Proteomics Genomics Proteomics Modeling Pathways Modeling Pathways Pharmaco- Pharmaco- Clinical Clinical genomics genomics Functional Chem- Functional Chem- Genomics informatics Genomics informatics Graphic modified from original courtesy of Sun Microsystems
IT Challenges Genomics Genomics Proteomics Proteomics VLDB VLDB VLDB BioSystems BioSystems (100s of TBs TBs) ) (100s of (100s of TBs) Chem- Chem- informatics informatics Load Store Aggregate Search Collaborate Match Mine Visualize
Oracle Platform Genomics Genomics Proteomics Proteomics BioSystems BioSystems Chem- Chem- informatics informatics • Distributed Queries • Unlimited Scalability • Incremental Updates • Reliability (RAC) • XML Data Types/Searches • Security • iFS/collaboration • Workflow • Data Mining • Text searches • Extensible Indexing • Portal • Partitioning & parallel computing • Images &Video
Integrated NYC Spatial Architecture Spatially Enabled Business Applications GIS Specialist Systems Environmental Logistics Management Management Core Spatial & Business Transportation Data Repository Financial Management Crime Monitoring Citizen Portal DPW Services Asset Maintenance Topographic/Raster Cadastre Health & Social Criminal Justice Geo-coded Address Services Street Center Lines Assets Environmental Transport Education Health Planning Health/Social services Education Crime
Managing All the Data in an e- Enterprise Spatial Data Object Relational Data Field Employee Employee Emplo Emplo EXsdfe EXsdfe EXs EXs Abcd Abcd Documents XML Multimedia Prospects Messages Customers Infrastructure
Shell International: Web enabled GIS provides browser based access to users of corporate and geo- spatial data from the Oracle RDBMS and Spatial databases in one integrated window
Spatial Database Technology: Manage Location & Structure Data
Oracle9 i Spatial Capabilities Spatial Indexing Spatial Data Types Oracle Fast Access to Native Spatial Data Spatial Spatial Data Management in the DBMS Spatial Access Through SQL SELECT STREET_NAME FROM ROADS, COUNTIES WHERE SDO_RELATE(road_geom, county_geom, ‘ MASK=ANYINTERACT QUERYTYPE=WINDOW ’ ) = ‘ TRUE ’ AND COUNTYNAME= ‘ PASSAIC ’ ;
Vector Map Data in Oracle Tables Fisher Circle Coop Court 85th St. Road ROAD_ID NAME SURFACE LANES LOCATION 1 Pine Cir. Asphalt 4 2 2nd St. Asphalt 2 3 3rd St. Asphalt 2
Sub-surface Geological Analysis
Raster/Vector Mapping
How Spatial Data Is Stored Data type Geographic coordinates
Performing Location Query in Oracle9i Example:What are the nearest post offices to my office? + Station B K1Y 2C4 3 km SQL> SELECT P.Post_Office_Name, P.Address 2> FROM Post_Offices P, 163 Island Park Dr. 3> Address_Master A 4> WHERE K1Y 2C3 5> A.St_Address =‘163 Island Park Dr.’ 6> and A.City = ‘Ottawa’ 7> AND MDSYS.SDO_WITHIN_DISTANCE( Main Street 8> A.Location, P.Location, + Station P 9> ‘distance=3’) = ‘TRUE’; K1Y 2C3
Jphone J-Navi Launch May 2000 Oracle Spatial Platform Powers: • Worlds 1st Live Map Delivery to Phone • Over 1M color maps delivered per day • Vector/Raster Maps generated dynamically • Avg. Query Processing 200ms • Download time: Max 2 seconds • 30,000 user sessions per hour • 17M business listing & national map data • Java Servlet Technology • Prototype to Lauch: 6 Months • Unprecedented scalability, reliability & flexibility KDDI & DoCoMo: similar model
Extensible Database Framework Optimizer Extensibility Query Engine Index Engine Type Manager
Dealing with large data volumes � How large is large ? 100 ’ s of thousands is normal – Millions is interesting – 10 ’ s of millions is serious – 100 ’ s of millions is large – � What is the problem with large volumes ? They mean big structures – � Cumbersome to manage Long operations – � Data reload, refresh � Index rebuilds
Partitioning: Divide and Conquer Two reasons for partitioning For manageability For performance � Break large problems into � Query parallelism manageable pieces � Partition elimination � Can load / rebuild individual partitions � Can load / rebuild multiple partitions concurrently � Can partition tables, or indexes, or both Also spatial indexes – � Transparent to applications!
Oracle9i Spatial Features � Spatial Reference System � Spatial Operators � Versioning/Long Transactions � Linear Referencing � Quadtree/R-tree index Parallel Index create � � Geodetic Support � Spatial Aggregates � Topology * � Raster/Grid Management * � Spatial Data Mining * * Planned Release 10i
Life Sciences Data Management Trends
Expanding Data Storage Needs “ To meet the scientific 500TB goals we believe we 450TB need to add around 400TB 80 - 100TB of storage 350TB each year for the next 300TB 5 years ” 250TB 200TB Data Storage P. Butcher, Today 150TB The Sanger Centre 100TB 50TB 0 1994 1995 1996 1997 1998 Oct-1999 Jan-01 2002 2003 2004 2005 2006 Apr-2000 Nov-2001
Increasing Computational Load Computational Load x Multiplier Rising real costs or Genetic Data analytical triage 8x per 18 months Moore’s Law 2x per 18 months Time Source: Sun Microsystems Life Sciences marketing collateral
What does DBMS technology bring? 1. Access and storage of vast quantities of life science data from a variety of sources 2. High throughput loading, indexing, processing and update of information 3. Data integration from a variety of sources 4. Scalability and reliability problems 5. Find patterns & insights through queries, analyses and data mining 6. Collaboration & security challenges
Recommend
More recommend
