On-line Fusion of Functional Knowledge Within Distributed Sensor Networks Dominik Fisch, Bernhard Sick Research Group “Computationally Intelligent Systems” University of Applied Sciences Deggendorf www.cis-research.de 11th Colloquium of the DFG Priority Program 1183 “Organic Computing” October 7./8. 2010 Munich Fisch, Sick Knowledge Fusion October 2010 1 / 20
Brief Status Report 1 Emergence Measurement and Knowledge Exchange 2 New Emergence Measurement Techniques 3 Experimental Results 4 Artificial Data Real-World Data (Intrusion Detection) Conclusion and Outlook 5 Fisch, Sick Knowledge Fusion October 2010 2 / 20
Brief Status Report Brief Status Report – 1 Collaboration of intelligent systems (e.g., teams of robots, smart sensor networks, software agents) by exchanging classification rules communication How is the local environ- learned rules ment observed? ( functional knowledge ) How does a node react on certain observations? Fisch, Sick Knowledge Fusion October 2010 3 / 20
Brief Status Report Brief Status Report – 2 Research Issues 2009/2010 Theory: Comparison of our probabilistic classifier to some other functionally equivalent paradigms (Elsevier Information Sciences, 2010) Extension of this classifier for time series classification (IEEE Tr. Knowledge and Data Engineering, 2011) Divergence-based techniques for emergence measurement (IEEE SASO 2010, Best Paper Award) Autonomous assessment of the interestingness of classification rules (ICAART 2011) Knowledge fusion based on Bayesian parameter estimation techniques Application: Collaborative Intrusion Detection and Anomaly Detection (Elsevier Information Sciences, 2010; IEEE Tr. Dependable and Secure Computing, 2011; IFIP BICC 2010) Toolbox for the OC community Fisch, Sick Knowledge Fusion October 2010 4 / 20
Brief Status Report Brief Status Report – 2 Research Issues 2009/2010 Theory: Comparison of our probabilistic classifier to some other functionally equivalent paradigms (Elsevier Information Sciences, 2010) Extension of this classifier for time series classification (IEEE Tr. Knowledge and Data Engineering, 2011) Divergence-based techniques for emergence measurement (IEEE SASO 2010, Best Paper Award) Autonomous assessment of the interestingness of classification rules (ICAART 2011) Knowledge fusion based on Bayesian parameter estimation techniques Application: Collaborative Intrusion Detection and Anomaly Detection (Elsevier Information Sciences, 2010; IEEE Tr. Dependable and Secure Computing, 2011; IFIP BICC 2010) Toolbox for the OC community Fisch, Sick Knowledge Fusion October 2010 4 / 20
Emergence Measurement and Knowledge Exchange Emergence Measurement and Knowledge Exchange – 1 Why are we interested in emergence measurement? What is the relation of emergence measurement and knowledge exchange? Emergence Measurement: Detecting unexpected state changes of a (e.g., self-organizing) system. Knowledge Exchange: Detecting observations that cannot be “explained” by a given model (e.g., classifier) and realizing the need to acquire new knowledge (e.g., asking other agents for classification rules). Common: need for a kind of situation-awareness Fisch, Sick Knowledge Fusion October 2010 5 / 20
Emergence Measurement and Knowledge Exchange Emergence Measurement and Knowledge Exchange – 2 Fisch, Sick Knowledge Fusion October 2010 6 / 20
New Emergence Measurement Techniques Motivation of a New Technique The chicken farm example Emergence: property of the system that is irreducible to the constituent parts of the system Goal: quantitative measurement of emergence Figure: [Mnif & M¨ uller-Schloer, 2006] Fisch, Sick Knowledge Fusion October 2010 7 / 20
New Emergence Measurement Techniques Motivation of a New Technique The chicken farm example Emergence: property of the system that is irreducible to the constituent parts of the system Goal: quantitative measurement of emergence Figure: [Mnif & M¨ uller-Schloer, 2006] Fisch, Sick Knowledge Fusion October 2010 7 / 20
New Emergence Measurement Techniques Quantitative Emergence How to detect emergence? Selection of attributes Creation of a model of the current situation Comparison to the model of a past situation Solution (assuming that attributes are given) Use probabilistic models to represent the system state Use probabilistic measures to capture state changes Fisch, Sick Knowledge Fusion October 2010 8 / 20
New Emergence Measurement Techniques Discrete Entropy Difference (DED) – 1 First proposed by Mnif and M¨ uller-Schloer (2006) Each dimension represents an attribute of the system Categorization of continuous attributes Calculate the (discrete) entropy for each attribute Use the difference between two entropy values as indicator for emergent behavior 5.0 5.0 5.0 4.5 4.5 4.5 4.0 4.0 4.0 3.5 3.5 3.5 3.0 3.0 3.0 2.5 2.5 2.5 2.0 2.0 2.0 1.5 1.5 1.5 1.0 1.0 1.0 0.5 0.5 0.5 0.0 0.0 0.0 -0.5 -0.5 -0.5 -1.0 -1.0 -1.0 -1.5 -1.5 -1.5 -2.0 -2.0 -2.0 -2.5 -2.5 -2.5 -3.0 -3.0 -3.0 -3.5 -3.5 -3.5 -4.0 -4.0 -4.0 -4.5 -4.5 -4.5 -5.0 -5.0 -5.0 - 5 - 4 - 3 - 2 - 1 0 1 2 3 4 5 - 5 - 4 - 3 - 2 - 1 0 1 2 3 4 5 - 5 - 4 - 3 - 2 - 1 0 1 2 3 4 5 Fisch, Sick Knowledge Fusion October 2010 9 / 20
New Emergence Measurement Techniques Discrete Entropy Difference (DED) – 2 Practical applications have continuous attributes, too Problems: ◮ Categorization of attributes ◮ Correlation between attributes is ignored (but may be desired depending on application) 5.0 5.0 4.5 4.5 4.0 4.0 3.5 3.5 3.0 3.0 2.5 2.5 2.0 2.0 1.5 1.5 1.0 1.0 0.5 0.5 0.0 0.0 -0.5 -0.5 -1.0 -1.0 -1.5 -1.5 -2.0 -2.0 -2.5 -2.5 -3.0 -3.0 -3.5 -3.5 -4.0 -4.0 -4.5 -4.5 -5.0 -5.0 - 5 - 4 - 3 - 2 - 1 0 1 2 3 4 5 - 5 - 4 - 3 - 2 - 1 0 1 2 3 4 5 Fisch, Sick Knowledge Fusion October 2010 10 / 20
New Emergence Measurement Techniques Measurement Create model p for a time step t p Create model q for a later time step t q Measure the “difference” of the models Fisch, Sick Knowledge Fusion October 2010 11 / 20
New Emergence Measurement Techniques New Approach: Modelling Avoid categorization of continuous attributes Approach I (PW): non-parametric density model (Parzen window estimator, e.g. with Gaussian kernel) ◮ No assumptions on the underlying distribution are made ◮ Computationally expensive ◮ Comparison of two densities via sampling Approach II (GMM): parameterized functional model (e.g. Gaussian mixture models for continuous attributes) ◮ Needs information about the underlying distribution ◮ Easy handling (faster, less computations) ◮ For Gaussians the comparison can be done analytically Fisch, Sick Knowledge Fusion October 2010 12 / 20
New Emergence Measurement Techniques New Approach: Hellinger Distance Measuring the distance between two probability densities: Hellinger distance (Hel) � Hel ( p, q ) = 1 − BC ( p, q ) � � Bhattacharyya coefficient: BC ( p, q ) = p ( x ) q ( x ) d x Hellinger distance is restricted to [0 , 1] The Hellinger distance measure is compatible to approach I (PW) and approach II (GMM) Fisch, Sick Knowledge Fusion October 2010 13 / 20
Experimental Results Artificial Data Comparison: DED – Hel Hel Hel DED X DED X DED Y DED Y 1.5 1.5 Emergence Measure Emergence Measure 1 1 Distribution Distribution switch switch 0.5 0.5 0 0 600 800 1000 1200 1400 1600 1800 2000 600 800 1000 1200 1400 1600 1800 2000 Time Time Cluster @ (0,0) Cluster @ (1,1) 5.0 5.0 5.0 5.0 5.0 5.0 4.5 4.5 4.5 4.5 4.5 4.5 4.0 4.0 4.0 4.0 4.0 4.0 3.5 3.5 3.5 3.5 3.5 3.5 3.0 3.0 3.0 3.0 3.0 3.0 2.5 2.5 2.5 2.5 2.5 2.5 2.0 2.0 2.0 2.0 2.0 2.0 1.5 1.5 1.5 1.5 1.5 1.5 1.0 1.0 1.0 1.0 1.0 1.0 0.5 0.5 0.5 0.5 0.5 0.5 0.0 0.0 0.0 0.0 0.0 0.0 -0.5 -0.5 -0.5 -0.5 -0.5 -0.5 -1.0 -1.0 -1.0 -1.0 -1.0 -1.0 -1.5 -1.5 -1.5 -1.5 -1.5 -1.5 -2.0 -2.0 -2.0 -2.0 -2.0 -2.0 -2.5 -2.5 -2.5 -2.5 -2.5 -2.5 -3.0 -3.0 -3.0 -3.0 -3.0 -3.0 -3.5 -3.5 -3.5 -3.5 -3.5 -3.5 -4.0 -4.0 -4.0 -4.0 -4.0 -4.0 -4.5 -4.5 -4.5 -4.5 -4.5 -4.5 -5.0 -5.0 -5.0 -5.0 -5.0 -5.0 - 5 - 4 - 3 - 2 - 1 0 1 2 3 4 5 - 5 - 4 - 3 - 2 - 1 0 1 2 3 4 5 - 5 - 4 - 3 - 2 - 1 0 1 2 3 4 5 - 5 - 4 - 3 - 2 - 1 0 1 2 3 4 5 - 5 - 4 - 3 - 2 - 1 0 1 2 3 4 5 - 5 - 4 - 3 - 2 - 1 0 1 2 3 4 5 Fisch, Sick Knowledge Fusion October 2010 14 / 20
Experimental Results Artificial Data DED – Hel with PW – Hel with GMM 0.7 Scenario: Hel PW Hel GMM 0.6 DED X DED Y Different processes generating 0.5 Emergence Measure 0.4 data 0.3 Drift One process (the topmost) start 0.2 begins to move ( concept drift ) 0.1 0 5.0 5.0 4.5 4.5 -0.1 4.0 4.0 3.5 3.5 3.0 3.0 -0.2 2.5 2.5 200 400 600 800 1000 1200 1400 1600 1800 2000 2.0 2.0 Time 1.5 1.5 1.0 1.0 0.5 0.5 0.0 0.0 With assumptions about the -0.5 -0.5 -1.0 -1.0 -1.5 -1.5 underlying distribution -2.0 -2.0 -2.5 -2.5 -3.0 -3.0 -3.5 -3.5 (Hel GMM), an earlier detection -4.0 -4.0 -4.5 -4.5 -5.0 -5.0 is possible - 5 - 4 - 3 - 2 - 1 0 1 2 3 4 5 - 5 - 4 - 3 - 2 - 1 0 1 2 3 4 5 Fisch, Sick Knowledge Fusion October 2010 15 / 20
Recommend
More recommend
