Data Fusion in Sensor Networks Hugh Durrant-Whyte ARC Federation - PowerPoint PPT Presentation

Data Fusion in Sensor Networks Hugh Durrant-Whyte ARC Federation Fellow, Research Director ARC Centre of Excellence for Autonomous Systems The University of Sydney, Australia hugh@cas.edu.au ARC=Australian version of NSF Slide 1 IPAM Sensor Networks Jan. 2007

Data Fusion in Autonomous Networks • Decentralised Data Fusion (DDF) • The DDF paradigm • Decentralised Bayes and information fusion • Fusion Challenges (ANSER II) • Learning mixed sensor features • Building general density models • Control Challenges (RCS-18) • Sensor and communication management • Information in system design • Future Challenges Slide 2 IPAM Sensor Networks Jan. 2007

Decentralised Data Fusion (DDF) Sensor • A set of Network Data Fusion Methods: Fusion • Ad Hoc Network Sensor • Fusion at Sensor/Platform • No Central Fusion Site Fusion Sensor • Fully Scalable Sensor • Decentralised Algorithms Fusion Using the Information Filter: Fusion Target states • For Target Tracking (different bandwidths) • For Cooperative Control • For Cooperative Navigation To other network components Slide 3 IPAM Sensor Networks Jan. 2007

Bayes and Information Fusion ( ) ( ) ( ) ∏ − = x Z x Z 1 x k k ( ) | . ( ) | ( ) | ( ) P k C P k P z k k Bayes: j j Posterior Prior Sensor Likelihoods • x (k) is the state at time k • z j (k) is the j th sensor observation at time k • Z k is the sequence of observations up to k Sensors contribute = Λ x x ( ( ) | ( )) ( ( )) P z k k k j j Likelihoods on x ( k ) Slide 4 IPAM Sensor Networks Jan. 2007

Data Fusion: Distributed Sensing Fusion Centre P( x ) n P( x | Z n )=C P( x ) Π Λ i ( x ) i=1 Λ 1 ( x ) Λ i ( x ) Λ n ( x ) Sensor 1 Sensor i Sensor n ...... ...... P( z 1 | x ) P( z i | x ) P( z n | x ) z 1 z i z n X Slide 5 IPAM Sensor Networks Jan. 2007

Data Fusion: Distributed Fusion Fusion Centre k k-1 n P( x k | Z n )=C P( x k-1 ) Π P i ( x k | z ) P( x k-1 | Z n ) i=1 P 1 ( x k | z ) P n ( x k | z ) P( x k | Z n ) P( x k | z 1 ) P( x k | z n ) Sensor n Sensor 1 ...... P( z 1 | x ) P( z n | x ) z 1 z n X Slide 6 IPAM Sensor Networks Jan. 2007

Data Fusion: Decentralised Fusion Fusion Centre Fusion Centre k k-1 k k-1 n n P( x k | Z n )=C P( x k-1 ) Π P i ( x k | z ) P( x k | Z n )=C P( x k-1 ) Π P i ( x k | z ) P( x k-1 | Z n ) P( x k-1 | Z n ) i=1 i=1 P 1 ( x k | z ) P n ( x k | z ) P( x k | z 1 ) P( x k | z n ) P( x k | Z n ) P( x k | Z n ) Sensor 1 Sensor n ...... P( z 1 | x ) P( z n | x ) z 1 z n X Slide 7 IPAM Sensor Networks Jan. 2007

Bayes and Information Fusion ( ) ( ) ( ) ∏ − = x Z x Z 1 x k k ( ) | . ( ) | ( ) | ( ) P k C P k P z k k Bayes: j j Log-Likelihood: Information, by construction, is additive ( ) ( ) ( ) ∑ − = + + x Z x Z 1 x k k ln ( ) | ln ( ) | ln ( ) | ( ) P k P k P z k k K j j [ ] = − x x ( ) log ( ) H E P Information: A measure of compactness: ⎡ ⎤ x z ( | ) P = − x z Mutual Information: a priori measure ( : ) log ⎢ ⎥ I E x ⎣ ⎦ ( ) of contribution to compaction: P Slide 8 IPAM Sensor Networks Jan. 2007

Bayes and Information Fusion ( ) ( ) ( ) ∏ − = x Z x Z 1 x k k ( ) | . ( ) | ( ) | ( ) P k C P k P z k k Bayes: j j Log-Likelihood: Information, by construction, is additive ( ) ( ) ( ) ∑ − = + + x Z x Z 1 x k k ln ( ) | ln ( ) | ln ( ) | ( ) P k P k P z k k K j j Gaussian Case: (Fisher or Canonical) Information Form − − = = i H R 1 z T y P 1 x ( ) ( ) ˆ ˆ ( | ) ( | ) ( | ) k k k k k k k k j j j j − = P − Y 1 = I H R 1 H T ( | ) ( | ) k k k k ( ) j k j j j Slide 9 IPAM Sensor Networks Jan. 2007

The Information Filter Observation updates are simple sums (unlike KF): ∑ = − + y y i ˆ ˆ ( | ) ( | 1 ) ( ) k k k k j k j ∑ = − + Y Y I ( | ) ( | 1 ) ( ) k k k k j k j Time/Structure updates are Dual to state (KF) Observation Updates : [ ] + = + + y y Ω y Y Bu ˆ ˆ ˆ ( 1 | ) ( | ) ( | ) ( | ) ( ) k k k k k k k k k + = − Y Y Ω Σ Ω T ( 1 | ) ( | ) ( ) ( ) ( ) k k k k k k k Slide 10 IPAM Sensor Networks Jan. 2007

DDF in Operation • Network communicates Information • Nodes fuse local observation and communicated Information Sensor • Channels communicate local information gain (mutual information) Fusion Sensor Channel Filters ∆ y qj (k|k) Sensor Node Fusion Sensor Channel Preprocess Manager ~ Fusion i (k) y i (k|k) Σ ∆ y pj (k|k) To other network components y i (k|k-1) Σ ∆ y pi (k|k) y i (k|k) Prediction Slide 11 IPAM Sensor Networks Jan. 2007

ANSERII: All-Source Bayesian Fusion • Full Bayes DDF for fusion of heterogeneous data from UAVs, UGVs, human and data base sources • Model general feature types; trees, buildings, dams, etc • Identify and label features, integrate human inferences • Real-time exploitation of network data by air, ground, human Slide 12 IPAM Sensor Networks Jan. 2007

Bayesian DDF Node Structure Sensor Node Preprocess z i (k) P i ( z | x ) P Density Likelihood and Feature Channel Filter Fitting Model Extraction P i ( z = z i (k)| x ) Q P( x k | Z k-1 , z i (k)) Observation Channel Channel Filter Update Manager (Multiplication) P( x k | Z k-1 , z i (k)) P( x k | Z k-1 ) P( x k | Z k ) Time Update Assimilation (Convolution) (Multiplication) P( x k | z Q , z P ) Slide 13 IPAM Sensor Networks Jan. 2007

Slide 14 Learning and Inference Process IPAM Sensor Networks Jan. 2007

Step I: Model Generation • NLDR on image patches • Find low-D representation for significant patches of interest • Use VBEM to find number and centre for mixture model s x z ( ) ( ) ( ) ( ) = z x s z x s x s s , , | , | P P P P Slide 15 IPAM Sensor Networks Jan. 2007

VBEM to Estimate Classifications • Need to estimate number and location of classes • Variational Bayes Procedures: • Simultaneous estimation of class number and centre • Can deal with order 10 5 state dimensions • Unsupervised/semi- supervised classification of sensory data • Automatic generation of Likelihood Function Slide 16 IPAM Sensor Networks Jan. 2007

Resulting Location Likelihood Models ANSER 2 • Mixture Model for Location Parameters and Likelihoods Slide 17 IPAM Sensor Networks Jan. 2007

Resulting Class Likelihood Models ( ) ( ) ( ) z x s x s s | , | P P P Slide 18 IPAM Sensor Networks Jan. 2007

Step II: Model Inference (Air) P ( z | x , s )= P ( z | x =[ x , s ]) is the required likelihood for inference IPAM Sensor Networks Jan. 2007

Step II: Model Inference (Ground) P ( z | x , s )= P ( z | x =[ x , s ]) is the required likelihood for inference Slide 20 IPAM Sensor Networks Jan. 2007

ANSER II Also Uses Data-Base and Human-derived Information • Human operator input: • Metric Information • Labels, Context • On-line estimation of “operator likelihood” • Hyperspectral data “node” Slide 21 IPAM Sensor Networks Jan. 2007

Slide 22 Component-Based Middleware for Deployment (ORCA) IPAM Sensor Networks Jan. 2007

Slide 23 Mission System Implementation System Video IPAM Sensor Networks Jan. 2007

Slide 24 ANSER II Common Operating Picture IPAM Sensor Networks Jan. 2007

Fusion: Future Challenges • Feature Modelling: Finding “x” • Generalising single-sensor feature models • Learning mixed-modality feature models • Learning and refining abstractions (MI?) • Probabilistic Fusion: Finding “P” • High-dimensional density estimation • Find a general and efficient density family, closed under fusion operations • Data association with general densities • Systems design: Finding “z” • Use information to do system design, assembly and reconfiguration Slide 25 IPAM Sensor Networks Jan. 2007

Mutual Information & Control • Mutual Information or information gain, is exactly what is communicated in the DDF • Can be exploited in sensor management, communications and platform control Channel Filters ∆ y qj (k|k) Sensor Node Channel Preprocess Manager ~ i (k) y i (k|k) Σ ∆ y pj (k|k) y i (k|k-1) Σ ∆ y pi (k|k) y i (k|k) Prediction Slide 26 IPAM Sensor Networks Jan. 2007

Example Cooperative Control • The trajectory that maximises information • Information shared (DDF) • Inherits DDF properties: • Scalability • Survivability Slide 27 IPAM Sensor Networks Jan. 2007

RCS-18: Future Cooperative UAVs • How best to use tactical UAV fleets ? • A list of candidate targets of interest • Coordinate a UAV fleet with mixed sensors to: • Locate, • identify and • prosecute targets • Demonstrate this Slide 28 IPAM Sensor Networks Jan. 2007

Set-up • DDF Enabled on all platforms • Mutual information on target location and IDs • A set of UAV manoeuvres: • Point-to-Point • Orbits • K-step look-ahead ϕ ω e R ∫ = ϕ I I d Orbit V ϕ s Slide 29 IPAM Sensor Networks Jan. 2007

Slide 30 Targets IPAM Sensor Networks Jan. 2007