Toward CEOS Water Portal 2.0 Yoshiyuki Kudo (JAXA/RESTEC) - PowerPoint PPT Presentation

Toward CEOS Water Portal 2.0 Yoshiyuki Kudo (JAXA/RESTEC) WGISS-35, INPE, Brazil

Background • Catalog creation and management has been labor-intensive – Diverse & inconsistent metadata from data partner servers (OPeNDAP, WFS, etc) – The portal is planned to become fully operational in another 3 years as part of DIAS 1 • Need of transition to labor-free/reduced architecture • Architecture approach based on 2 step search (much like CWIC, ESIP OpenSearch etc.) [1] http://www.editoria.u-tokyo.ac.jp/projects/dias/?locale=en 1

Architecture on Search & Access -Today-  1 2 CEOS Water Portal Data Centers 2 Data Access User (eg. OPeNDAP) OPeNDAP Servers -NASA 1 Search -GPCC(NOAA) (xQuery) -GLOWASIS -CEOP MOLTS Local XML Database -FLUXNET ingest WFS, etc Source metadata in native format -Catalog of datasets from all the data centers - Registered Use-cases Defining schema Creation of WaterPortal tailored-catalog Performed manually 2

Architecture on Search & Access -Planned-   1 2 3 CEOS Water Portal Database Water Portal Dataset DIFs IDN/GCMD 1 (For servers not in conformance to broker harvest functionality ） CEOS Water Portal -CEOP Gridded Model -CUAHSI UI component CEOS Water Portal catalog broker component 2 Data Centers (OSS broker eg. Geoportal) Granule-level User catalog OPeNDAP Servers -NASA Local XML Database 2 Step-search-enabled -GPCC(NOAA) OpenSearch servers -GLOWASIS 1 -CEOP MOLTS 2 -FLUXNET ISO19115/19139 -Registered Use cases (eg) NASA ECHO (CEOP,AWCISatellite,AWCIInsitu ) 3 Data Access (OPeNDAP etc.) WFS(GEMS/Water) etc. 3 *1 MWS: Metadata Web Service by IDN/GCMD

New architecture Open, distributed architecture • Based on 2 step search (Dataset search -> Granule search) • – Case1 (basic case) Dataset Search : MWS (Metadata Web Service by IDN/GCMD) • Granule Search : OpenSearch (CEOS Water Portal catalog broker component ） • – Case 2 (applicable to 2-step OpenSearch-enabled partner servers) Dataset Search : OpenSearch • Granule Search : OpenSearch • For dataset-level catalog, create and ingest DIFs for the entire water portal • datasets (except datasets of pattern 2) Use OSS (Open Source Software) for brokering the granule level catalog • – Harvest from each partner servers in an automated fashion – Candidate OSS : Esri Geoportal. New User Interface • – Search&access than drill-down – Category search by IDN/GCMD Science Keyword as well as ECV variables – Support free text search 4

Mechanism of Dataset->Granule Search Transition for Case 1 • Prearrangements – In DIFs, specify project=waterPortal and embed OSDD URL (OpenSearch Description Document) that is applicable to each specific dataset – Customize the catalog broker component to be able to recognize the dataset-specific OSDD URL and generate the OSDD.xml dynamically – Assure that a search query based on the OSDD URL template returns only the granules of the specific dataset (eg.by using a set of fixed query terms, or insert DIF’s EntryID into the broker component database/ response somehow) • How it works – Search IDN/GCMD using MWS with project=WaterPortal – Parse the response and get the dataset-specific OSDD – Generate and send OpenSearch query for granule-level search 5

Mechanism of Dataset->Granule Search Transition for Case 2 Much like Case 1 except it uses OpenSearch for dataset level search • Much like the way ESIP Discovery Cluster proposes • 6

Hurdles to overcome 1. Performance of catalog broker OSS Search response time • Holding and managing tens of millions of granule records • Customizable ? • 2. DIF creation Can we (portal) make and ingest the partners’ dataset DIFs on our own ? • (IDN/GCMD policy ?) Effective methodology for creating the large number of DIFs • Granularity of a dataset (DIF) • (eg. CEOP MOLTS ECMWF 2007 Precipitation Dataset, or CEOP MOLTS ECMWF Dataset) 3. Sustainability of current portal’s services Subsetting by variable / time (ESIP Discovery group seems to be working on • OpenSearch -> OPeNDAP transition mechanizm) 7

Feasibility study first • We will do feasibility study through prototyping the new architecture this fiscal year. • Transition to the new architecture will happen the following year, if it yields satisfying results. 8

Need support from WGISS • Endorsement from IDN/GCMD team about leveraging IDN/GCMD in the presented way. • We would like to solicit the involvement and support of Dr. Chris Lynnes, CEOS Water Portal NASA representative, for helping us establish good communication with Geoportal development team on this ambitious challenge as well as for comments on any aspect. 9

• Comments ? 10