Understanding and Modeling of WiFi Signal Based Human Activity - PowerPoint PPT Presentation

Understanding and Modeling of WiFi Signal Based Human Activity Recognition Wei Wang † , Alex X. Liu †‡ , Muhammad Shahzad ‡ ,Kang Ling † , Sanglu Lu † † Nanjing University, ‡ Michigan State University September 8, 2015 1/24

Motivation Modeling Design Experiments Conclusions Motivation • WiFi signals are available almost everywhere and they are able to monitor surrounding activities. 2/24

Motivation Modeling Design Experiments Conclusions Motivation • WiFi signals are available almost everywhere and they are able to monitor surrounding activities. 2/24

Motivation Modeling Design Experiments Conclusions Problem Statment WiFi based Activity Recognition • Using commercial WiFi devices to recognize human activities. Wireless router Wireless signal reflection Laptop Advantages � Work in dark � Better coverage � Less intrusive to user privacy � No need to wear sensors 3/24

Motivation Modeling Design Experiments Conclusions Problem Statment WiFi based Activity Recognition • Using commercial WiFi devices to recognize human activities. Wireless router Wireless signal reflection Laptop Advantages � Work in dark � Better coverage � Less intrusive to user privacy � No need to wear sensors 3/24

Motivation Modeling Design Experiments Conclusions Problem Statment WiFi based Activity Recognition • Using commercial WiFi devices to recognize human activities. Wireless router Wireless signal reflection Laptop Advantages � Work in dark � Better coverage � Less intrusive to user privacy � No need to wear sensors 3/24

Motivation Modeling Design Experiments Conclusions Problem Statment WiFi based Activity Recognition • Using commercial WiFi devices to recognize human activities. Wireless router Wireless signal reflection Laptop Advantages � Work in dark � Better coverage � Less intrusive to user privacy � No need to wear sensors 3/24

Motivation Modeling Design Experiments Conclusions Problem Statment WiFi based Activity Recognition • Using commercial WiFi devices to recognize human activities. Wireless router Wireless signal reflection Laptop Advantages � Work in dark � Better coverage � Less intrusive to user privacy � No need to wear sensors 3/24

Motivation Modeling Design Experiments Conclusions Challenges • Measurement from commercial devices are noisy and have unpredictable carrier frequency offsets • Needs robust and accurate models to extract useful infor- mation from measurements 75 70 CSI 65 60 11 11.5 12 12.5 Time (seconds) 4/24

Motivation Modeling Design Experiments Conclusions Understanding Multipath Key observations Sender • Multipaths contain both static component and dynamic com- LoS path d k (0) ponent Reflected by d k (0) • Each path has different phase wall Reflected by • Phases determine the ampli- body tude of the combined signal Wall Receiver 5/24

Motivation Modeling Design Experiments Conclusions Understanding Multipath Sender Q Dynamic LoS path Component Static d k (0) component Reflected by d k (0) wall Combined Reflected by body I Wall Receiver 6/24

Motivation Modeling Design Experiments Conclusions Understanding Multipath Sender Q Dynamic LoS path Component d k (t) LoS path Static component Reflected by wall Combined Reflected by body I Wall Receiver 6/24

Motivation Modeling Design Experiments Conclusions Understanding Multipath Sender Q Combined Dynamic Component d k (t) LoS path Static Reflected by component wall Reflected by body I Wall Receiver 6/24

Motivation Modeling Design Experiments Conclusions Understanding Multipath Interpreting CSI amplitude Q • Phases of paths are deter- mined by path length Dynamic • Path length change of one Component Static wavelength gives phase component change of 2 π • Frequency of amplitude Combined change can be converted to movement speed I 7/24

Motivation Modeling Design Experiments Conclusions CSI-Speed Model How accurate is it? • Wave length → 5 ∼ 6 cm in 5 GHz band 200 CSI power 150 100 50 2.5 3 3.5 4 4.5 5 5.5 6 6.5 7 Time (seconds) Waveform with regular moving speed CSI amplitude changes are close to sinusoids 8/24

Motivation Modeling Design Experiments Conclusions CSI-Speed Model How accurate is it? • Wave length → 5 ∼ 6 cm in 5 GHz band 1 200 0.8 CSI power 150 0.6 CDF 0.4 100 0.2 0 50 0 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 2.5 3 3.5 4 4.5 5 5.5 6 6.5 7 Measurement error (meters) Time (seconds) Moving distance measurement error Waveform with regular moving speed Average distance measurement CSI amplitude changes are error of 2.86 cm close to sinusoids 8/24

Motivation Modeling Design Experiments Conclusions CSI-Speed Model How robust is it? • Robust over different multipath conditions and movement directions • Linear combination of multipath do not change frequency 0.25 running 0.2 Probability walking sitting down 0.15 0.1 0.05 0 0.2 0.4 0.6 0.8 1 1.2 1.4 Estimated speed (m/s) Speed distribution of different activities in different environments 9/24

Motivation Modeling Design Experiments Conclusions CSI-Activity Model Activities are characterized by • Movement speeds • Change in movement speeds • Speeds of different body components 15 40 20 10 20 5 CSI CSI CSI 0 0 0 −5 −20 −20 −10 −15 −40 −40 2 2.5 3 3.5 4 0.5 1 1.5 2 0 0.5 1 1.5 2 Time (seconds) Time (seconds) Time (seconds) Falling Walking Sitting down 10/24

Motivation Modeling Design Experiments Conclusions CSI-Activity Model • Use time-frequency analysis to extract features • Use HMM to characterize the state transitions of movements Walking Falling Sitting down 11/24

Motivation Modeling Design Experiments Conclusions CSI-Activity Model • Build one HMM model for each activity • Determine states based on observations in waveform pat- terns • State durations and relationships are captured by transition probabilities State 1 State 2 State 3 State 4 12/24

Motivation Modeling Design Experiments Conclusions System Architecture Activity CSI measurement data collection collection Noise reduction Online Model monitoring generation Activity detection and segmenting HMM training Feature extraction HMM HMM based activity Model recognition Monitoring records 13/24

Motivation Modeling Design Experiments Conclusions Data Collection N × M × 30 CSI streams 30 subcarriers 75 75 70 70 CSI CSI 65 65 60 60 11 11.5 12 12.5 11 11.5 12 12.5 Time (seconds) Time (seconds) 75 75 70 70 CSI CSI 65 65 60 60 M 11 11.5 12 12.5 11 11.5 12 12.5 Time (seconds) Time (seconds) 75 75 N 70 70 CSI CSI 65 65 60 60 11 11.5 12 12.5 11 11.5 12 12.5 Time (seconds) Time (seconds) 75 75 70 70 CSI CSI 65 65 60 60 11 11.5 12 12.5 11 11.5 12 12.5 Time (seconds) Time (seconds) 75 75 70 70 CSI CSI 65 65 60 60 11 11.5 12 12.5 11 11.5 12 12.5 Time (seconds) Time (seconds) 14/24

Motivation Modeling Design Experiments Conclusions Noise Reduction Correlation of CSI on different subcarriers • Subcarriers only differ slightly in wavelength • Subcarriers have the same set of paths, with different phases Wave length Wave length = 5.150214 cm = 5.149662 cm 312.5kHz Frequency 15/24

Motivation Modeling Design Experiments Conclusions Correlation in CSI Streams 16/24

Motivation Modeling Design Experiments Conclusions Noise Reduction Combines N × M × 30 subcarriers using PCA to detect time- varying correlations in signal 75 75 70 70 CSI CSI 65 65 60 11 11.5 12 12.5 11 11.5 12 12.5 Time (seconds) Time (seconds) Original Low-pass filter 10 5 CSI 0 −5 −10 11 11.5 12 12.5 Time (seconds) PCA 17/24

Motivation Modeling Design Experiments Conclusions Real-time Recognition • Activity detection - Use both the signal variance and correlation to detect pres- ence of activities • Feature extraction - Time-frequency analysis (DWT) • HMM model building - Eight activities Walking, running, falling, brushing teeth, sitting down, opening refrigerator, pushing, boxing - More than 1,400 samples from 25 persons as the training set 18/24

Motivation Modeling Design Experiments Conclusions Evaluation Setup • Commercial hardware with no modification - Transmitter: NetGEAR JR6100 Wireless Router - Receiver: Thinkpad X200 with Intel 5300 NIC • A single communicating pair is enough to monitor 450 m 2 open area • Measurement on UDP packets sent between the pair • Sampling rate 2,500 samples per second 19/24