KES: Knowledge Enabled Services for better EO Information Use - PowerPoint PPT Presentation

KES: Knowledge Enabled Services for better EO Information Use Andrea Colapicchioni Advanced Computer Systems Space Division a.colapicchioni@acsys.it

The problem � During the last decades, the satellite image catalogues have stored huge quantity of data � State of the art catalogues permit only to specify location, time of interest, metadata like platform, sensor, acquisition mode… IGARSS 2004 - Image Information Mining

The interpretation task � The interpretation of EO images requires � Fusion of data/information for better understanding of structures � Aggregation with existing knowledge specific to the application fields (at higher level) IGARSS 2004 - Image Information Mining

A little bit of history.. 2004 - 2006 1996 - 2000 2002 - 2004 KEO (ACS, DLR, GTD, CNES) IIM (DLR, ETH) KES (DLR, ACS) 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 1996 2006 2001 - 2002 2003 - 2004 KIM (ACS, DLR, ETH) KIMV (ACS, DLR) � 1996-2000 IIM (Image Information Mining) (http://isis.dlr.de/mining) � 2001: KIM (Knowledge Information Mining) (http://www.acsys.it:8080/kim) � 2002: KES (Knowledge Enabled Services) � 2003: KIMV (KIM Validation) � 2004: KEO (Knowledge-centred Earth Observation) IGARSS 2004 - Image Information Mining

KIM: Knowledge Driven Information Mining Primitive Primitive Features: - Texture Feature Images - Colour Extraction - Shape (at pixel level) 010010 010010 101101 101101 110010 110010 IGARSS 2004 - Image Information Mining

More in detail IGARSS 2004 - Image Information Mining

KIM Interactive learning IGARSS 2004 - Image Information Mining

From data to semantic Primitive Features: - Texture Features Images Primitive - Colour Interactive Feature - Shape Learning Extraction River 010010 010010 101101 101101 Forest 110010 110010 Cloud DATA SEMANTIC IGARSS 2004 - Image Information Mining

From KIM to KES (Knowledge Enabled Services) � Image interpretation is not a simple task. Each user needs a set of accessory data, as for example GIS layers or texts obtained through Internet. � Yet, the amount of available information makes searches a demanding and expensive task. � An environment where images are at the focal point, and where each user can navigate through a taxonomically structured knowledge, could be of extreme value. IGARSS 2004 - Image Information Mining

Other semantic Primitive Primitive Interactive Image Features Images Feature Learning Features extraction Text Features Text Interaction GIS Interaction GIS Features Layers DATA SEMANTIC IGARSS 2004 - Image Information Mining

KES: New interface IGARSS 2004 - Image Information Mining

Semantic Grouping � In KIM it is simple to define a feature by identifying a river through positive and negative examples. � However, the “river" might become part of a wider concept (e.g.: water). This should be implemented without retraining the system for all possible water types. IGARSS 2004 - Image Information Mining

Aggregated Features � In KES a new kind of grouping, called “aggregated features”, has been introduced. � In a similar way as in defining positive and negative examples on the image, it will be possible to define the concept “water” as: � Positive examples: � sea + river + lake + water reservoir � Negative examples: � streets + houses + mountains IGARSS 2004 - Image Information Mining

Semantic Grouping Primitive Primitive Aggregated Interactive Image Images Semantic Feature Learning Features Image Grouping extraction Features Features Descriptive G Text roupings Descriptive Groupings Text Interaction Features Descriptive Groupings GIS GIS Object Interaction Descriptive Layers Features Descriptor Groupings DATA SEMANTIC IGARSS 2004 - Image Information Mining

Ontology � Ontology is the specification of a conceptualisation. It can be related to a system (System Ontology, which can be domain independent and reused for different domains) or to a domain (Domain Ontology, specific for that domain). IGARSS 2004 - Image Information Mining

Domain Ontology � Domain ontology is the set of definitions and concepts pertaining and belonging to a specific domain (and shared by concerned people). � Different domains have generally different ontologies. As an example, a climatology expert could have a different vision of (and terms to describe) water compared to that of an oceanography expert. IGARSS 2004 - Image Information Mining

Taxonomy Domain n Domain 1 … Sub domain Sub domain Sub domain Aggregated Aggregated Image Feature Image Feature Text GIS Feature Feature Image Image Image Feature Feature Feature Text Text GIS GIS Layer Layer Image Image Image Image IGARSS 2004 - Image Information Mining

Data / Semantic / Ontology Aggregated Primitive Primitive Image Interactive Semantic Images Feature Image Features Learning Features Grouping extraction Features Descriptive G roupings Text Interaction Descriptive Text Groupings Features Domains D e s c r i p G t i r v o e u p i n g s GIS GIS Interaction Layers Object Descriptive Features Groupings Descriptor DOMAIN DATA SEMANTIC ONTOLOGY ONTOLOGY IGARSS 2004 - Image Information Mining

Implicit and explicit knowledge transfer � The KIM prototype permits to associate weighted combinations of primitive features to image features. While defining features, the user explicitly transfers knowledge to the system, which is stored and made available to the same or other users. � In addition there is a further type of knowledge (implicit) that could be discovered: if the user domain of interest is known, by observing the user interactions with the system during search and browsing, it is possible to infer the data of likely user interest and link it with the pertinence domain. IGARSS 2004 - Image Information Mining

Knowledge transfer Urban Factories Areas Roads Cities Sea Temperature Lake Census Map IKONOS Image Roads ERS Image Political Landsat Facts Image Climate Meris Algae Image Changes Blooming IGARSS 2004 - Image Information Mining

Knowledge Graph Spectral (feature) www.a.com Water www.a.com Hydrology River MERIS Image Water GIS www.b.com www.a.com Ikonos Image River GIS IGARSS 2004 - Image Information Mining

Knowledge Discovery Exploring this graph means discovering the user's knowledge IGARSS 2004 - Image Information Mining

KIMV (KIM Validation) A practical case study: Automatic cloud classification in MERIS images

Cloud Cover Characterization MERIS Level 1 – Reduced Resolution “cloud” KIMV System IGARSS 2004 - Image Information Mining

Cloud cover characterization Map of Cloud object IGARSS 2004 - Image Information Mining

MAP Object Extraction Binarized, closed MAP of “cloud” label IGARSS 2004 - Image Information Mining

Tiling I MG Vote Shape 12345 60 (x1,y1)-(x2,y2)…. 12345 20 (x1, y1)-(x2,y2)… IGARSS 2004 - Image Information Mining

MASS IGARSS 2004 - Image Information Mining

Linux Cluster � Cluster nodes: 10 Dell 1750, dual Xeon 3 GHz KVM Switch Switch 4400 Giga Switch 6 7 P a c k e t 1 2 3 5 4 6 7 8 9 1 0 1 1 1 2  1 1 2 S a t u t P a c k e t s 3 1 1 1 4 2 1 5 3 1 6 4 7 5 1 8 1 6 9 1 2 7 8 9 0 2 2 1 1 0 1 2 2 2 1 1 3 4 2 M o d u l 1 e Module 2 5 3 1 P o w e r / S e l f 4 2 6 T e s t � Database and Application Server: t u a t S s 1 1 3 4 5 1 1 6 1 7 1 8 2 9 1 0 2 1 2 2 2 2 4 3 7 8 Packet - Green = Full Duplex Yellow = Half Duplex 3 1 8 1 9 1 4 2 a t S - s t u Green Yellow = 10Mbps = 100Mbps flashing on = enabled, link ok disabled = 3C17203 Super Stack 3 Rackable Monitor & Keyboard (1U) Dell 6600, 4 Xeon 2.8 GHz P o w e r 1 7 5 0 E d g e X E O N P o w e r E d g e X E O N 0 1 7 5 PowerEdge 1750 (x 5) P o w e r E d g e X E O N 0 1 7 5 P o w e r E d g e 1 7 5 0 E O X N PowerEdge 2650 PowerVault 1 1 2 2 6 6 0 F 3 3 4 5 5 4 6 6 Power Vault (14x146 GB) 3 6 G B 1 0 K 3 6 GB r p m C F 0 K 1 p m r 6 GB 3 F C 1 0 3 6 K r GB p m C F 1 0 GB p m 3 6 r K F C 1 0 3 6 G K r p m B F C 1 0 3 6 GB K r p m F C 3 6 GB 0 K 1 r p m F C r p 6 GB 0 K 1 3 m F C 0 K 1 6 GB 3 p m r F C 3 6 GB 1 0 K r p m F C 1 0 3 6 K p m GB r C F 3 6 1 0 K GB r p m C F 1 0 3 6 K GB r p m C F 1 0 K r p m F C RACK 01 IGARSS 2004 - Image Information Mining

System Context Internet MASS Internet PORTAL Internet User Client KI M SYSTEM Direct Connection KIM Server MASS Expert user TOOLBOX Direct Connection JDBC Parallel Ingestion Chain Ingestion daemon Ingestion chain monitor WEB SERVICE RDBMS Workflow Engine MERIS ARCHIVE FACILITY IGARSS 2004 - Image Information Mining

KIM prototype � http://www.acsys.it:8080/kim � Andrea Colapicchioni � a.colapicchioni@acsys.it Visit the ACS stand for a demo IGARSS 2004 - Image Information Mining