Understanding and Modeling of WiFi Signal Based Human Activity - PowerPoint PPT Presentation

Understanding and Modeling of WiFi Signal Based Human Activity Recognition Wei Wang † , Alex X. Liu †‡ , Muhammad Shahzad ‡ ,Kang Ling † , Sanglu Lu † † Nanjing University, ‡ Michigan State University September 8, 2015 1/24

Motivation Modeling Design Experiments Conclusions Motivation • WiFi signals are available almost everywhere and they are able to monitor surrounding activities. 2/24

Motivation Modeling Design Experiments Conclusions Problem Statment WiFi based Activity Recognition • Using commercial WiFi devices to recognize human activities. Wireless router Wireless signal reflection Laptop Advantages � Work in dark � Better coverage � Less intrusive to user privacy � No need to wear sensors 3/24

Motivation Modeling Design Experiments Conclusions Problem Statment WiFi based Activity Recognition • Using commercial WiFi devices to recognize human activities. Wireless router Wireless signal reflection Laptop Advantages � Work in dark � Better coverage � Less intrusive to user privacy � No need to wear sensors 3/24

Motivation Modeling Design Experiments Conclusions Problem Statment WiFi based Activity Recognition • Using commercial WiFi devices to recognize human activities. Wireless router Wireless signal reflection Laptop Advantages � Work in dark � Better coverage � Less intrusive to user privacy � No need to wear sensors 3/24

Motivation Modeling Design Experiments Conclusions Problem Statment WiFi based Activity Recognition • Using commercial WiFi devices to recognize human activities. Wireless router Wireless signal reflection Laptop Advantages � Work in dark � Better coverage � Less intrusive to user privacy � No need to wear sensors 3/24

Motivation Modeling Design Experiments Conclusions Problem Statment WiFi based Activity Recognition • Using commercial WiFi devices to recognize human activities. Wireless router Wireless signal reflection Laptop Advantages � Work in dark � Better coverage � Less intrusive to user privacy � No need to wear sensors 3/24

OFDM PH Y B asics Motivation Modeling Design Experiments Conclusions 24/24

at time t p - #Tx carrier for an antenna stream information of each sub- amplitude and phase Frequency Response) - Every h entry - CFR (Channel q - #Rx CSI measurement taken every frame Commercial Off-the-Shelf Cards provide 30 sub-carriers CSI - Channel State Information Motivation Modeling Design Experiments Conclusions 24/24

internal state changes (power and rate adaption) CFO - Channel Frequency Offset - 802.11n 5GHz channel - sub- Noise - electromagnetic interference; frequency carrier frequency can drift by up-to 100 kHz from central Motivation Modeling Design Experiments Conclusions Challenges • Measurement from commercial devices are noisy and have unpredictable carrier frequency offsets • Needs robust and accurate models to extract useful infor- mation from measurements 75 70 CSI 65 60 11 11.5 12 12.5 Time (seconds) 4/24

Motivation Modeling Design Experiments Conclusions Understanding Multipath Key observations Sender • Multipaths contain both static component and dynamic com- LoS path d k (0) ponent Reflected by d k (0) • Each path has different phase wall Reflected by • Phases determine the ampli- body tude of the combined signal Wall Receiver 5/24

Motivation Modeling Design Experiments Conclusions Understanding Multipath Sender Q Dynamic LoS path Component Static d k (0) component Reflected by d k (0) wall Combined Reflected by body I Wall Receiver 6/24

Motivation Modeling Design Experiments Conclusions Understanding Multipath Sender Q Dynamic LoS path Component d k (t) LoS path Static component Reflected by wall Combined Reflected by body I Wall Receiver 6/24

Motivation Modeling Design Experiments Conclusions Understanding Multipath Sender Q Combined Dynamic Component d k (t) LoS path Static Reflected by component wall Reflected by body I Wall Receiver 6/24

Motivation Modeling Design Experiments Conclusions Understanding Multipath Interpreting CSI amplitude Q • Phases of paths are deter- mined by path length Dynamic • Path length change of one Component Static wavelength gives phase component change of 2 π • Frequency of amplitude Combined change can be converted to movement speed I 7/24

= path length change = 1/2 mov. distance CSI Waveform for a movement with change and then multiply by wavel e nght - Hilbert Transform used to find phase rangefinder - Ground truth determined using laser Measurement of path length change 0.8m pathlength change Motivation Modeling Design Experiments Conclusions CSI-Speed Model How accurate is it? • Wave length → 5 ∼ 6 cm in 5 GHz band • Steel plate of diameter 30 cm moving along a straight line 200 CSI power 150 100 50 2.5 3 3.5 4 4.5 5 5.5 6 6.5 7 Time (seconds) Waveform with regular moving speed CSI amplitude changes are close to sinusoids 8/24

Motivation Modeling Design Experiments Conclusions Motivation Modeling Design Experiments Conclusions CSI-Speed Model CSI-Speed Model How accurate is it? • Wave length → 5 ∼ 6 cm in 5 GHz band • Steel plate of diameter 30 cm moving along a straight line 1 200 0.8 CSI power 150 0.6 CDF 0.4 100 0.2 0 50 0 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 2.5 3 3.5 4 4.5 5 5.5 6 6.5 7 Measurement error (meters) Time (seconds) Moving distance measurement error Waveform with regular moving speed Average distance measurement CSI amplitude changes are error of 2.86 cm close to sinusoids 8/24

and the speeds of path length change o Linear combination of multipath do not change frequency o Robust over different multipath conditions and movement directions (88% accuracy with 0-0.61-1 separation) o Motivation Modeling Design Experiments Conclusions CSI-Speed Model How robust is it? • CFR amplitude - linear combination of the reflected paths 0.25 running 0.2 Probability walking sitting down 0.15 0.1 0.05 0 0.2 0.4 0.6 0.8 1 1.2 1.4 Estimated speed (m/s) Speed distribution of different activities in different environments 9/24

Motivation Modeling Design Experiments Conclusions CSI-Activity Model Activities are characterized by • Movement speeds • Change in movement speeds • Speeds of different body components 15 40 20 10 20 5 CSI CSI CSI 0 0 0 −5 −20 −20 −10 −15 −40 −40 2 2.5 3 3.5 4 0.5 1 1.5 2 0 0.5 1 1.5 2 Time (seconds) Time (seconds) Time (seconds) Falling Walking Sitting down 10/24

Motivation Modeling Design Experiments Conclusions CSI-Activity Model • Denoise CSI Values (PCA based) • Use time-frequency analysis to extract features (DWT) • Use HMM to characterize the state transitions of movements Walking Falling Sitting down 11/24

Motivation Modeling Design Experiments Conclusions CSI-Activity Model • Build one HMM model for each activity • Determine states based on observations in waveform pat- terns • State durations and relationships are captured by transition probabilities State 1 State 2 State 3 State 4 12/24

Motivation Modeling Design Experiments Conclusions System Architecture Activity CSI measurement data collection collection Noise reduction Online Model monitoring generation Activity detection and segmenting HMM training Feature extraction HMM HMM based activity Model recognition Monitoring records 13/24

Motivation Modeling Design Experiments Conclusions Data Collection N × M × 30 CSI streams 30 subcarriers 75 75 70 70 CSI CSI 65 65 60 60 11 11.5 12 12.5 11 11.5 12 12.5 Time (seconds) Time (seconds) 75 75 70 70 CSI CSI 65 65 60 60 M 11 11.5 12 12.5 11 11.5 12 12.5 Time (seconds) Time (seconds) 75 75 N 70 70 CSI CSI 65 65 60 60 11 11.5 12 12.5 11 11.5 12 12.5 Time (seconds) Time (seconds) 75 75 70 70 CSI CSI 65 65 60 60 11 11.5 12 12.5 11 11.5 12 12.5 Time (seconds) Time (seconds) 75 75 70 70 CSI CSI 65 65 60 60 11 11.5 12 12.5 11 11.5 12 12.5 Time (seconds) Time (seconds) 14/24

Motivation Modeling Design Experiments Conclusions Noise Reduction Correlation of CSI on different subcarriers • Subcarriers only differ slightly in wavelength • Subcarriers have the same set of paths, with different phases • Principal Component Analysis (PCA) to filter noises Wave length Wave length = 5.150214 cm = 5.149662 cm 312.5kHz Frequency 15/24

Motivation Modeling Design Experiments Conclusions Correlation in CSI Streams Correlation of CSI on different subcarriers • Noise in principal component 1 is discarded, next 5 are kept • CSI "peaks" are red, "valleys" are blue 16/24

Motivation Modeling Design Experiments Conclusions Noise Reduction Combines N × M × 30 subcarriers using PCA to detect time- varying correlations in signal 75 75 70 70 CSI CSI 65 65 60 11 11.5 12 12.5 11 11.5 12 12.5 Time (seconds) Time (seconds) Original Low-pass filter 10 5 CSI 0 −5 −10 11 11.5 12 12.5 Time (seconds) 2nd PCA Component 17/24