Unmanned Aerial Vehicles Using WiFi Chitra R. Karanam and Yasamin - PowerPoint PPT Presentation

3D Through-Wall Imaging With Unmanned Aerial Vehicles Using WiFi Chitra R. Karanam and Yasamin Mostofi Department of Electrical and Computer Engineering University of California Santa Barbara 16th ACM/IEEE International Conference on Information Processing in Sensor Networks, Pittsburgh, PA USA

Sensing with RF Signals Counting people Elderly fall detection Smart home Through-wall imaging 2

Imaging and Robotics Why imaging? Why robots? • • Search and rescue Soon to be part of our society • • Archaeological exploration Can go to hazardous places • • Surveillance Form autonomous networks • • Mapping Allow for autonomous and optimized antenna positioning 3

Robotic Through-Wall 3D Imaging with WiFi RSSI X1.5 Speed 4

Robotic Through-Wall 3D Imaging with WiFi RSSI State-of-the-art: 2D Imaging • Y. Mostofi. "Cooperative wireless-based obstacle/object mapping and see-through capabilities in robotic networks." IEEE Transactions on Mobile Computing 12.5 (2013): 817-829. • S. Depatla, L. Buckland, and Y. Mostofi. "X-ray vision with only wifi power measurements using Rytov wave models." IEEE Transactions on Vehicular Technology 64.4 (2015): 1376-1387. Challenges in 3D Imaging • Considerably more under-determined system • 3D path planning and design • Localization of air vehicles considerably more challenging 5

Outline • Proposed 3D Through-wall Imaging Pipeline − System Modeling and Sparse Signal Processing − Markov Random Field Modeling and Belief Propagation • Route Design for UAVs: Robotic Paths in 3D • Experimental Testbed: UAVs and Google Tangos • 3D Imaging Experimental Results • Conclusions 6

Outline • Proposed 3D Through-wall Imaging Pipeline − System Modeling and Sparse Signal Processing − Markov Random Field Modeling and Belief Propagation • Route Design for UAVs: Robotic Paths in 3D • Experimental Testbed: UAVs and Google Tangos • 3D Imaging Experimental Results • Conclusions 7

Measurement Model Using Wentzel-Kramers-Brillouin (WKB) wave approximation Shadowing WiFi RSSI measurement Path loss Modeling error 𝑄 𝑆 𝒒 𝒋 , 𝒓 𝒋 = 𝑄 𝑄𝑀 𝒒 𝒋 , 𝒓 𝒋 + 𝛿 ෍ 𝑒 𝑗𝑘 𝜃 𝑗𝑘 + 𝜂 𝒒 𝒋 , 𝒓 𝒋 𝑘 Cell dimension TX pos. RX pos. = 𝑄 𝑄𝑀 𝒒 𝒋 , 𝒓 𝒋 + 𝛿 ෍ 𝜃 𝒔 𝒌 Δ𝑒 + 𝜂 𝒒 𝒋 , 𝒓 𝒋 𝑘∈ℒ(𝑞 𝑗 ,𝑟 𝑗 ) Decay rate of signal Cells along 𝑗 𝑢ℎ measurement line due to object at 𝒔 𝑘 8

Measurement Model (cont.) 𝑄 𝑗 ≡ 𝑄 𝑆 𝒒 𝒋 , 𝒓 𝒋 − 𝑄 𝑄𝑀 (𝒒 𝒋 , 𝒓 𝒋 ) ≅ ෍ 𝜃 𝒔 𝒌 𝛿 Δ𝑒 𝑘∈ℒ 𝑞 𝑗 ,𝑟 𝑗 3D domain 𝑏 11 𝑏 12 𝑏 1𝑂 𝜃(𝒔 1 ) 𝑄 1 ⋯ 𝑏 21 𝑏 22 𝑏 2𝑂 𝑄 2 𝜃(𝒔 2 ) = ⋮ ⋱ ⋮ ⋮ ⋮ 𝑏 𝑁1 𝑏 𝑁2 ⋯ 𝑏 𝑁𝑂 𝑄 𝑁 𝜃(𝒔 𝑂 ) Observation vector: 𝑸 Vector of signal decay rate: 𝑷 Cells along a measurement line if 𝑘 𝑢ℎ cell is on 𝑗 𝑢ℎ measurement line 𝑏 𝑗𝑘 = ቊ 1 0 otherwise 𝑸 = 𝑩𝑷 Linear measurement model 9

Sparse Signal Processing 𝑸 = 𝑩 × 𝑷 Linear measurement model 𝑁 × 1 𝑁 × 𝑂 𝑂 × 1 • M ≪ N - severely under-determined system • Real spaces – generally sparse in spatial variations • Regularization – Total Variation minimization 𝑛𝑗𝑜𝑗𝑛𝑗𝑨𝑓 𝑈𝑊 𝑷 , 𝑡𝑣𝑐𝑘𝑓𝑑𝑢 𝑢𝑝 𝑸 = 𝑩𝑷 𝐽 𝑗+1,𝑘,𝑙 − 𝐽 𝑗,𝑘,𝑙 𝑂 𝐽 𝑗,𝑘+1,𝑙 − 𝐽 𝑗,𝑘,𝑙 𝑈𝑊 𝑷 = ෍ 𝐸 𝑛 𝑷 2 , 𝐸 𝑛 𝑷 = , 𝐽 = 3D matrix version of 𝑷 . where, 𝐽 𝑗,𝑘,𝑙+1 − 𝐽 𝑗,𝑘,𝑙 𝑛=1 2D slice of 3D Ground-truth image image from TV Min. 10 Challenging to image the area!

Markov Random Field Modeling • Model spatial dependencies 𝑍 𝑗 − TV min. intensity observed at 𝑗 𝑢ℎ node • Decision at each cell depends on − Observed intensity from TV min. − Decision at neighboring cells 𝑌 𝑗 − Binary label at 𝑗 𝑢ℎ node • MRF model: 𝑄 𝑌 𝑗 = 𝑦 𝑗 𝑌 𝑘 = 𝑦 𝑘 , ∀ 𝑘 ≠ 𝑗 ) = 𝑄(𝑌 𝑗 = 𝑦 𝑗 | 𝑌 𝑘 = 𝑦 𝑘 , ∀ 𝑘 ∈ 𝒪 𝑗 ) • Using Hammersley-Clifford theorem: Neighborhood of node 𝑗 𝑌 𝑗 ∈ 0, 1 , 0 ≤ 𝑍 𝑗 ≤ 1 𝑄 𝒀 = 𝒚 𝒁 = 𝒛) ∝ exp −𝐹 𝒚, 𝒛 , Cost function definition 𝐹 𝒚, 𝒛 = ෍ Φ 𝑗 𝑦 𝑗 , 𝑧 𝑗 + ෍ Φ 𝑗𝑘 (𝑦 𝑗 , 𝑦 𝑘 ) Φ 𝑗 𝑦 𝑗 , 𝑧 𝑗 = ൝ 1 − 𝑦 𝑗 boundary 𝑦 𝑗 − 𝑧 𝑗 2 else 𝑗 𝑗,𝑘 ∈Ɛ 2 Φ 𝑗𝑘 𝑦 𝑗 , 𝑦 𝑘 = 𝑦 𝑗 − 𝑦 𝑘 Total cost Data cost Discontinuity cost 11

Markov Random Field Modeling (cont.) • Interested in finding solution 𝒀 = 𝒚 that maximizes 𝑄 𝒀 = 𝒚 𝒁 = 𝒛) ∝ exp −𝐹 𝒚, 𝒛 • Challenging: NP hard problem in loopy graphs • Sum-product Loopy Belief Propagation − Computes marginal probability at each cell 𝑄(𝑌 𝑗 = 𝑦 𝑗 |𝒁 = 𝒛) − Estimated label: ො 𝑦 𝑗 = arg max 𝑄(𝑌 𝑗 = 𝑦 𝑗 |𝒁 = 𝒛) 𝑦 𝑗 − Distributed approximated solution and computationally efficient 2D slice of 3D 2D slice of 3D image - MRF and LBP Ground-truth image image from TV Min. 12

Outline • Proposed 3D Through-wall Imaging Pipeline − System Modeling and Sparse Signal Processing − Markov Random Field Modeling and Belief Propagation • Route Design for UAVs: Robotic Paths in 3D • Experimental Testbed: UAVs and Google Tangos • 3D Imaging Experimental Results • Conclusions 13

UAV Path Planning - Challenges Example 3D scenario 3 example x-z plane cross- sections at different y’s • Different planes contain different information about the area • Need to design routes that capture all possible information 14

UAV Path Planning – Route Design X1.5 Speed Top view X1.5 Speed Top view Front view • Parallel routes (angles: 0°, 90°, 45° and 135° ) • 4 angles for diverse perspectives of area Front view • Horizontal routes capture x-y variations at one height • Sloped routes capture variations in z with small number of measurements 15

Outline • Proposed 3D Through-wall Imaging Pipeline − System Modeling and Sparse Signal Processing − Markov Random Field Modeling and Belief Propagation • Route Design for UAVs: Robotic Paths in 3D • Experimental Testbed: UAVs and Google Tangos • 3D Imaging Experimental Results • Conclusions 16

Experimental Testbed Requirements: • 3D path planning • Accurate localization • Coordination between UAVs • RSSI measurements 17

Experimental Testbed (cont.) WiFi router Raspberry Pi on TX UAV on RX UAV TX UAV RX UAV Wireless comm. link GOOGLE GOOGLE RX Tango TX Tango TANGO TANGO Wireless comm. link Wireless comm. link Remote PC 18

Experimental Testbed (cont.) Localization Route Control • Way-points: position goals • Google Tango • Cameras and IR sensors • Uses features of environment • Real-time accurate positioning • Streams position information to UAV for route control • RMSE of localization: 0.067 m Coordination WiFi RSSI • TX UAV: WiFi router Reached • RX UAV: way-point 3? − Raspberry Pi − WLAN card Not yet. • Location-stamped RSSI Wait for me measurements 19

Outline • Proposed 3D Through-wall Imaging Pipeline − System Modeling and Sparse Signal Processing − Markov Random Field Modeling and Belief Propagation • Route Design for UAVs: Robotic Paths in 3D • Experimental Testbed: UAVs and Google Tangos • 3D Imaging Experimental Results • Conclusions 20

Experimental Scenario: Two-Cube Area 21

3D Through-Wall Imaging: Experimental Results Two-Cube Area • 3D high-quality imaging with WiFi • Empty and occupied places imaged well • Accurate localization of center of top block • Variation along z is captured in reconstruction 22

Experimental Scenario: L-shape Area 23

3D Through-Wall Imaging: Experimental Results L-shape Area • 3D high-quality imaging with WiFi • Empty and occupied places imaged well • Accurate localization of center of top block • Variation along z is captured in reconstruction 24

Conclusions • 3D Through-Wall Imaging − Aerial vehicles and WiFi − Sparsity, MRF modeling and Loopy Belief Propagation for binary map of area • Efficient Informative Route Design for UAVs • Experimental Testbed − Google Tangos for localization and coordination − Motion planning and control • Experimental Results − Accurate through-wall imaging of 3D objects using very few measurements 25

Thank you! This work is funded by NSF CCSS award # 1611254 26