Using Sensors to Detect Landmines CoVar Applied Technologies The - PowerPoint PPT Presentation

Using Sensors to Detect Landmines CoVar Applied Technologies The CoVar Group 1 Duke University MPI, 2016 The CoVar Group (MPI 2016) Using Sensors to Detect Landmines MPI, 2016 1 / 33

Overview Confidence Sensor Score Alarm Set Map Data Alarm Set Sensors collect data over a given region Confidence maps are generated over data typically by target detection algorithms Alarms are placed using information provided by confidence map Placement of alarms is compared with ground truth locations of known targets Detection performance is estimated The CoVar Group (MPI 2016) Using Sensors to Detect Landmines MPI, 2016 2 / 33

A Topological Data Analysis 5 threshold 1 4.5 4 threshold 2 3.5 3 threshold 3 C ( x ) 2.5 2 1.5 1 0.5 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 x Lower the threshold → Find region topologically connected For each region sufficiently large Keep the prominent peak Lower the remaining peaks The CoVar Group (MPI 2016) Using Sensors to Detect Landmines MPI, 2016 3 / 33

Confidence Map - Before & After (a) (b) Figure: (a) Before (b) After The CoVar Group (MPI 2016) Using Sensors to Detect Landmines MPI, 2016 4 / 33

Alarm Set - Before & After 0.7 0.6 0.5 0.4 CoVar Algorithm Pd Topological Algorithm 0.3 0.2 0.1 0 0 0.01 0.02 0.03 0.04 0.05 0.06 0.07 FAR The CoVar Group (MPI 2016) Using Sensors to Detect Landmines MPI, 2016 5 / 33

Denoise by local averaging Motivation: Noise levels are different at different places. Algorithm: For each point ( x , y ), the confidence level is C ( x , y ). Define the noise 1 � level at ( x , y ) as noise ( x , y ) = D ( x , y ) C ( s , t ) dsdt , where m ( D ( x , y )) D ( x , y ) is the rectangle centered at ( x , y ) and m() is the area. The new confidence level is defined as C new ( x , y ) = C ( x , y ) / noise ( x , y ). Before denoise After denoise The CoVar Group (MPI 2016) Using Sensors to Detect Landmines MPI, 2016 6 / 33

Denoise by local averaging ROC-FAR curve with 5000 alarms The CoVar Group (MPI 2016) Using Sensors to Detect Landmines MPI, 2016 7 / 33

Alarm Aggregation: Hierarchical Clustering 1.5 1 0.5 CoVar AlarmSet Reduced AlarmSet 0 − 0.5 − 1 − 1.5 0 200 400 600 800 1000 1200 1400 Find the similarity among alarms 0.7 0.6 based on their distance. 0.5 Group the alarms into clusters . 0.4 Pd Keep only the alarm with the 0.3 CoVar Algorithm Clustering highest confidence in each cluster. 0.2 0.1 0 0 0.01 0.02 0.03 0.04 0.05 0.06 0.07 FAR The CoVar Group (MPI 2016) Using Sensors to Detect Landmines MPI, 2016 8 / 33

Iterative Centroid Alarm Set Generation Possible Concern: Placing alarms at the location of a local maximum does not consider asymmetry in the confidence map about the local maximum. Possible Solution: Centroid Location Scheme Assume digger will (eventually) dig a circle of radius r about a given alarm For a given alarm, consider the neighborhood of radius r about the location of the alarm Consider the confidence map, c ( x , y ), as a mass-density function and calculate the centroid over the neighborhood Use centroid location as new alarm location Repeat process until centroid location is fixed The CoVar Group (MPI 2016) Using Sensors to Detect Landmines MPI, 2016 9 / 33

Example The CoVar Group (MPI 2016) Using Sensors to Detect Landmines MPI, 2016 10 / 33

Diligent Digger Sensor based algorithm performs well for easily identifiable targets Low confidence alarms perform no better than randomly placed alarms Can we design a Diligent Digger that performs better than the low confidence data informed alarms? The CoVar Group (MPI 2016) Using Sensors to Detect Landmines MPI, 2016 11 / 33

Recall 0.8 0.7 0.6 0.5 Pd 0.4 0.3 0.2 0.1 roc far roc quiver 0 0 0.2 0.4 0.6 0.8 1 1.2 FAR The CoVar Group (MPI 2016) Using Sensors to Detect Landmines MPI, 2016 12 / 33

Naive Diggers Random and grid uninformed alarm sets developed to test benchmark performance Confidences randomly assigned for alarm locations Grid alarm placement superior to random placement on average The CoVar Group (MPI 2016) Using Sensors to Detect Landmines MPI, 2016 13 / 33

Augmented Algorithm Replace low confidence alarms with uninformed alarms based on a confidence threshold Identify threshold that informed alarms perform no better than random alarms Can we perform better below this threshold? The CoVar Group (MPI 2016) Using Sensors to Detect Landmines MPI, 2016 14 / 33

Diligent Digger 1) Sensor based algorithm positions 2) Remove least confident alarms 3) Augment remaining alarms with uniform grid 4) Remove redundant grid alarms 1 2 3 4 The CoVar Group (MPI 2016) Using Sensors to Detect Landmines MPI, 2016 15 / 33

Diligent Digger The CoVar Group (MPI 2016) Using Sensors to Detect Landmines MPI, 2016 16 / 33

Alternative method for alarm generation using a combinatorial approach. Discretize search area (road) into n x × n y squares. (First we treat n y = 1, i.e. a one dimensional road.) Treat bombs as rectangles that are b x × b y squares. Treat each non-overlapping placement of n b bombs as equally likely. Below, search area is 42 × 8, three bombs that are 4 × 2. The CoVar Group (MPI 2016) Using Sensors to Detect Landmines MPI, 2016 17 / 33

Place bomb randomly along road with a fixed signal strength of µ . The CoVar Group (MPI 2016) Using Sensors to Detect Landmines MPI, 2016 18 / 33

Add Standard Normal noise at each point along the road. The CoVar Group (MPI 2016) Using Sensors to Detect Landmines MPI, 2016 19 / 33

Sweep a bomb width interval along the road. Find the one with the most area. The CoVar Group (MPI 2016) Using Sensors to Detect Landmines MPI, 2016 20 / 33

Sweep a bomb width interval along the road. Find the one with the most area. The CoVar Group (MPI 2016) Using Sensors to Detect Landmines MPI, 2016 21 / 33

Sweep a bomb width interval along the road. Find the one with the most area. The CoVar Group (MPI 2016) Using Sensors to Detect Landmines MPI, 2016 22 / 33

Sweep a bomb width interval along the road. Find the one with the most area. The CoVar Group (MPI 2016) Using Sensors to Detect Landmines MPI, 2016 23 / 33

(new) AREA method places alarm at midpoint of interval that maximizes area. (CoVar) PEAK method places alarm at the highest peaks. The CoVar Group (MPI 2016) Using Sensors to Detect Landmines MPI, 2016 24 / 33

Bomb signal strength is Normal ( µ, 1) Background noise is Normal (0 , 1) For each µ ∈ [0 , 5], we compare the average performance of the AREA and PEAK alarm generation algorithms over 1000 simulations With one bomb and one alarm, AREA performed better than PEAK for all fixed (known) sizes of bomb The CoVar Group (MPI 2016) Using Sensors to Detect Landmines MPI, 2016 25 / 33

2d Area Method Difference even more pronounced in higher dimension and with increased number of bombs Persists regardless of whether clustered peaks are suppressed in alarm set Right: Three 1x3 bombs in 2x10 domain The CoVar Group (MPI 2016) Using Sensors to Detect Landmines MPI, 2016 26 / 33

2d Area Method The CoVar Group (MPI 2016) Using Sensors to Detect Landmines MPI, 2016 27 / 33

As we let n x and n y approach infinity, the discrete distribution of bomb locations approaches the following continuous one. The CoVar Group (MPI 2016) Using Sensors to Detect Landmines MPI, 2016 28 / 33

The CoVar Group (MPI 2016) Using Sensors to Detect Landmines MPI, 2016 29 / 33

The CoVar Group (MPI 2016) Using Sensors to Detect Landmines MPI, 2016 30 / 33

The CoVar Group (MPI 2016) Using Sensors to Detect Landmines MPI, 2016 31 / 33

Hypothesis Test of Independence Confidence Maps vs. True Targets Hypothesis Test of Independence: H 0 : Alarm pdf ⊥ ⊥ Truth pdf vs. H 1 : Alarm pdf � ⊥ Truth pdf ⊥ Table: Hypothesis Test of Independence: Confidence Maps vs. True Targets n Test Mean P-Value Median P-Value 5000 Distance Correlation 0.2476 0.0100 1000 Distance Correlation 0.3774 0.1300 200 Cross-Match 0.8782 1000 Cross-Match 0.1286 2100 Cross-Match 0.719339 4000 Cross-Match 0.2886 5000 Cross-Match 0.3085 7000 Cross-Match 0.3373 The CoVar Group (MPI 2016) Using Sensors to Detect Landmines MPI, 2016 32 / 33

Figure: DC Test ( n = 5000): P-Values of various samples with replacement The CoVar Group (MPI 2016) Using Sensors to Detect Landmines MPI, 2016 33 / 33

Summary Filtered confidence map data in a variety of different ways Given this filtering, came up with different ways of placing alarms Improved up random performance for higher FAR levels Proposed a more concise algorithm scoring method Thank you CoVar! The CoVar Group (MPI 2016) Using Sensors to Detect Landmines MPI, 2016 33 / 33