Aspects of Pervasive Sensing: Perception and Security from ambient - PowerPoint PPT Presentation

Aspects of Pervasive Sensing: Perception and Security from ambient noise Stephan Sigg Department of Communications and Networking Aalto University, School of Electrical Engineering stephan.sigg@aalto.fi TU-BS, 27.04.2017

Cheap collaboration Radio Vision Security from ambient signals Stephan Sigg April 27, 2017 2 / 31

Stephan Sigg April 27, 2017 3 / 31

Stephan Sigg April 27, 2017 3 / 31

Feedback-based distributed adaptive beamforming Stephan Sigg April 27, 2017 4 / 31

Feedback-based distributed adaptive beamforming ◮ Weak multimodal fitness function ◮ Single local = global optimum j G a i n ) + γ i cos( ) t i f + γ i ) ϕ 2 π 2 π f t γ i ( ϕ ( j j i e e ϕ i 1 cos( ) i ϕ j(2 π f t + γ i ) e −δ i δ i Stephan Sigg April 27, 2017 5 / 31

Stephan Sigg April 27, 2017 6 / 31

Stephan Sigg April 27, 2017 6 / 31

Stephan Sigg April 27, 2017 6 / 31

Feedback-based distributed adaptive beamforming ◮ Weak multimodal fitness function ◮ Single local = global optimum j G a i n ) + γ i cos( ) t i f + γ i ) ϕ 2 π 2 π f t γ i ( ϕ ( j j i e e ϕ i 1 cos( ) i ϕ j(2 π f t + γ i ) e −δ i δ i Stephan Sigg April 27, 2017 7 / 31

Feedback-based distributed adaptive beamforming Stephan Sigg April 27, 2017 8 / 31

Feedback-based distributed adaptive beamforming Stephan Sigg April 27, 2017 8 / 31

Feedback-based distributed adaptive beamforming Stephan Sigg April 27, 2017 8 / 31

Cheap collaboration Radio Vision Security from ambient signals Stephan Sigg April 27, 2017 9 / 31

g Stephan Sigg April 27, 2017 10 / 31

Stephan Sigg April 27, 2017 10 / 31

RF-sensing for environmental perception ◮ Multi-path propagation ◮ Reflection ◮ Signal superimposition ◮ Blocking of signal paths ◮ Scattering ◮ Doppler Shift ◮ Signal Phase ◮ Fresnel effects Stephan Sigg April 27, 2017 11 / 31

RF-based activity recognition Sensewaves Video Stephan Sigg April 27, 2017 12 / 31

RF-based device-free activity recognition g Crawling n g n i d i k n a l t a S W empty L y i n g Stephan Sigg April 27, 2017 13 / 31

RF-based device-free activity recognition g Crawling n g n i d i k n a l t a S W empty L y i n g Stephan Sigg April 27, 2017 13 / 31

– Video – Stephan Sigg April 27, 2017 14 / 31

Cheap collaboration Radio Vision Security from ambient signals Stephan Sigg April 27, 2017 15 / 31

Motivation 6 Stephan Sigg April 27, 2017 16 / 31

Motivation Trust and proximity We will use audio as a source of common information in proximity 6 Stephan Sigg April 27, 2017 16 / 31

Security from environmental stimuli Real-time implementation on android mobile phones a a Stephan Sigg, et al., AdhocPairing: Spontaneous audio-based secure device pairing for Android mobile devices, IWSSI 2012 ◮ Hardware noise cancellation on some phones ◮ Hardware originated synchronisation offset

Audio-based ad-hoc secure pairing 1 ◮ Use audio to generate secret key ◮ high Entropy, fuzzy cryptography, case studies, attack scenarios Hamming distance in created fingerprints Percentage of tests in one test run that passed at >5% for Kuiper KS p−values (loud audio source in 1.5m and 3m) Median percentage of identical bits in fingerprints Fingerprints created for matching audio samples 1.01947 (confidence value at α = 0.03) Fingerprints created for non−matching audio samples 1.01 0.8 Percentage of passed tests 0.99 0.75 0.7 0.97 0.65 0.95 0.6 0.93 0.92053 (confidence value at α = 0.03) 0.55 0.91 Only music Only clap Only speak Only snap Only whistle 0.5 Clap Music Snap Speak Whistle 0 2 4 6 8 10 12 14 16 18 20 Audio sequence class Test run 1S. Sigg et al., Secure Communication based on Ambient Audio, IEEE Transactions on Mobile Computing Stephan Sigg April 27, 2017 19 / 31

Secure pairing from noisy data possible codewords C possible messages X c ′ x C c Decoding Encoding Stephan Sigg April 27, 2017 20 / 31

Device-to-Device Authentication Stephan Sigg April 27, 2017 21 / 31

Accelerometer Reading Acceleration [m/s 2 ] 5 0 − 5 0 1 2 3 4 5 6 7 Time [s] ◮ Accelerometer reading on z-axis only Stephan Sigg April 27, 2017 22 / 31

Rotated Signal Acceleration [m/s 2 ] 20 10 0 0 1 2 3 4 5 6 7 Time [s] ◮ Orientation relative to ground using Madgwick’s Algorithm z y x ◮ Notice influence of gravity g g Stephan Sigg April 27, 2017 23 / 31

Noise-Reduced Signal Acceleration [m/s 2 ] 5 0 − 5 0 1 2 3 4 5 6 7 Time [s] ◮ Apply a bandpass filter to keep frequencies between 0.5 and 12 Hz Stephan Sigg April 27, 2017 24 / 31

Gait-Cycle Detection Acceleration [m/s 2 ] 5 0 − 5 0 1 2 3 4 5 6 Time [s] ◮ Partition data into gait cycles ◮ Resample gait cycles to equal length ◮ Calculate average gait cycle Stephan Sigg April 27, 2017 25 / 31

Quantization Acceleration [m/s 2 ] Acceleration [m/s 2 ] Acceleration [m/s 2 ] 5 5 5 0 0 0 − 5 − 5 − 5 Cycle Average Cycle 1 0 0 1 ◮ Average gait cycle overlaid on each original gait cycle ◮ 4 bits per cycle Stephan Sigg April 27, 2017 26 / 31

Quantization Acceleration [m/s 2 ] 5 0 − 5 a) 1001 0100 1001 1010 1010 1001 0101 0110 b) 1001 0100 1001 1010 1010 1001 0101 0110 c) 0111 1000 1001 0101 1000 1100 1011 1000 ◮ Average gait cycle overlaid on each original gait cycle ◮ 4 bits per cycle Stephan Sigg April 27, 2017 27 / 31

Comparison between Locations Acceleration [m/s 2 ] 5 0 − 5 forearm: 0111 1000 1001 0101 1000 1100 1011 1000 Acceleration [m/s 2 ] 5 0 − 5 waist: 0110 1000 1001 0001 1001 1001 1100 1010 Stephan Sigg April 27, 2017 28 / 31

Evaluation 1 0 . 8 Similarity 0 . 6 0 . 4 0 . 2 0 m y t d n h t m s s d a i g e h r i r a a o e i a h s h r w h b e c t e r - p a o p f r Inter-body u t n I Stephan Sigg April 27, 2017 29 / 31

Cheap collaboration Radio Vision Security from ambient signals Stephan Sigg April 27, 2017 30 / 31

Thank you! Stephan Sigg stephan.sigg@aalto.fi Stephan Sigg April 27, 2017 31 / 31