Relationship between Motion Vectors of Vection-Induced Image and Multivariate Biosignals under Visual Tasks Tohru Kiryu 1 , Yoko Nanbo 1 , Naoki Kobayasi 2 , and Takehiko Bando 3 1 Graduate School of Science and Technology, Niigata University, Niigata, Japan 2 NTT Cyber Space Laboratories, Yokosuka, Japan 3 Graduate School of Medical and Dental Sciences, Niigata University, Niigata, Japan Relationship between Motion Vectors of Vection-Induced Image and Multivariate Biosignals under Visual Tasks ， BSI2002, Como, Italy ， ， ， ， June 27, 2002
Background Background Engineering Side Medical Side Evaluation of Image Evaluation of Stress by Quality by SN Ratio Biosignals However, … However… Influence of image on Relationship with image human has not yet been has not been evaluated discussed Relationship between Image and Biosignals? Relationship between Motion Vectors of Vection-Induced Image and Multivariate Biosignals under Visual Tasks ， BSI2002, Como, Italy ， ， ， June 27, 2002 ，
Overview Overview distant-range 30 motion[pixel] left 0 right -30 20 70 [sec] Local Motion Vector Motion at Each Local Section Correlation? RR Interval Correlation? 0.8 10 motion[pixel] 0 0.6 70 20 -40 20 70 [sec] Global Motion Vector Motion of Camera Control zoom, pan, and tilt Relationship between Motion Vectors of Vection-Induced Image and Multivariate Biosignals under Visual Tasks ， BSI2002, Como, Italy ， ， ， June 27, 2002 ，
Purpose Purpose 1. Feature extraction from Multi-Biosignals that might contain useful information. → PCA Score 2. Quantitative representation of Image by Motion Vectors. → Time-Frequency Representation 3. Relationship between the Structure of Image and Biosignals. → Correlation 3. Searching the Structure of Image that affects human. → Simulation by CG 4. Design of Total Evaluation System → Contents Search Engine etc. Relationship between Motion Vectors of Vection-Induced Image and Multivariate Biosignals under Visual Tasks ， BSI2002, Como, Italy ， ， ， ， June 27, 2002
Approach Approach Biosignals Video Image BP ECG Resp. Image Feature Extraction RR Int. LPF Motion Vector PCA Arrangement Feature Extraction ， Arrangement Time-Series of Motion Vector Time-Series of PCA Score Correlation at each Block Extraction of ANA Info. Relationship between Motion Vectors of Vection-Induced Image and Multivariate Biosignals under Visual Tasks ， BSI2002, Como, Italy ， ， ， June 27, 2002 ，
Experiments Experiments Biosignals vehicle and bodily sensation video (Mountain Bike) - ECG ： chest-lead - Blood Pressure ： tonometry method. 80-inch Screen - Respiration : tube type sensor, at the chest & abdominal regions. - Perspiration ： capsule type sensor, at the thenar of left hand. - Blood Flow ： laser doppler, at the thenar of left hand. 2 [m] Subjects - Five healthy elderly subjects (from 52 to 71 yrs. old) - Six elderly subjects with mild hypertension or diabetes mellitus (5 males and 1 female from 50 to 71 yrs. old) School of Medicine, Niigata University (Nov.13-14,1999) Relationship between Motion Vectors of Vection-Induced Image and Multivariate Biosignals under Visual Tasks ， BSI2002, Como, Italy ， ， ， June 27, 2002 ，
Multivariate Biosignals Biosignals Multivariate 5min. Respiration R-R interval HF LF/HF Persipiration Blood flow Head motion(R/L) Head motion(A/P) Head motion(U/D) speech diving Mountain bike Racing car Car race Bike race Relationship between Motion Vectors of Vection-Induced Image and Multivariate Biosignals under Visual Tasks ， BSI2002, Como, Italy ， ， ， ， June 27, 2002
Arrangement by PCA Arrangement by PCA Subject A RR Int. BP Resp. 1.2 0.2 0.8 0.8 -0.2 0.6 20 70 20 70 20 70 time[sec] Z PCA Score x 1 2 = ϕ + ϕ + ϕ ϕ ： eigen vector Z x x x Direction ratio ϕ ϕ ϕ 1 11 1 12 2 13 3 : : = ϕ + ϕ + ϕ 11 12 13 Z x x x x 2 21 1 22 2 23 3 1 Z = ϕ + ϕ + ϕ Z x x x 2 Z x 3 31 1 32 2 33 3 3 3 1st PCA Score ( z 1 ) 2nd PCA Score ( z 2 ) 3rd PCA Score ( z 3 ) 4 4 4 -4 –4 -4 20 70 20 70 20 70 time[sec] Relationship between Motion Vectors of Vection-Induced Image and Multivariate Biosignals under Visual Tasks ， BSI2002, Como, Italy ， ， ， June 27, 2002 ，
TFRs of PCA Score Time-Series of PCA Score Time-Series TFRs Subject A Rest 0.8 0.8 Frequency[Hz] Frequency[Hz] 0.7 0.7 0.6 0.6 z 2 0.5 0.5 z 1 0.4 0.4 0.3 0.3 0.2 0.2 0.1 0.1 0 10 20 30 40 50 0 10 20 30 40 50 time[sec] time[sec] Image-stress 0.8 0.8 Frequency[Hz] Frequency[Hz] 0.7 0.7 0.6 0.6 z 1 z 2 0.5 0.5 0.4 0.4 0.3 0.3 0.2 0.2 0.1 0.1 20 30 40 50 60 70 20 30 40 50 60 70 time[sec] time[sec] Relationship between Motion Vectors of Vection-Induced Image and Multivariate Biosignals under Visual Tasks ， BSI2002, Como, Italy ， ， June 27, 2002 ， ，
Frequency Component around 0.1Hz Frequency Component around 0.1Hz Resting Image Task Subject BP Z 2 BP Z 2 ○ ○ A ○ ○ ○ ○ B elderly subjects ○ ○ ○ C with mild D NA NA NA NA hypertension or diabetes mellitus E NA NA NA NA F NA NA NA NA ○ ○ O ○ ○ ○ P healthy elderly subjects ○ ○ Q ○ ○ ○ R ○ ○ ○ S ・ frequency component around 0.1Hz appeared. ・ increase of perspiration ・ report of ill-condition Influence on human was expected Relationship between Motion Vectors of Vection-Induced Image and Multivariate Biosignals under Visual Tasks ， BSI2002, Como, Italy ， ， ， June 27, 2002 ，
Feature Extraction of Image Feature Extraction of Image Image Image Compression ・ compression by the correlation in space (space frequency, DCT) ・ compression by the correlation between frames → Motion Vector ・ compression by the deviation of appearance for sign (entropy cording) Motion Vector 30 [ frame / sec ] Correlation to Biosignals? 1- frame Static image Relationship between Motion Vectors of Vection-Induced Image and Multivariate Biosignals under Visual Tasks ， BSI2002, Como, Italy ， ， ， June 27, 2002 ，
Local Motion & Global Motion Local Motion & Global Motion distant-range b 1,1 left motion[pixel] 30 0 right Block Matching Method -30 y y Local Motion Vector Motion at each Local Section x x l -th frame ( l -1)th frame pan [pixel] 10 pan 0 -10 -20 -30 screen -40 Global Motion Vector camera Motion of Camera Control tilt Relationship between Motion Vectors of Vection-Induced Image and Multivariate Biosignals under Visual Tasks ， BSI2002, Como, Italy ， ， ， June 27, 2002 ，
Time-Series of LMV Time-Series of LMV distant-range middle-range Local Motion Vector close-range 30 motion[pixel] 30 motion[pixel] left u p b 1,1 0 0 right down -30 -30 30 30 motion[pixel] motion[pixel] b 3,3 0 0 -30 -30 30 motion[pixel] 30 motion[pixel] 0 b 5,3 0 -30 -30 20 30 40 50 60 70 20 30 40 50 60 70 time[sec] time[sec] Relationship between Motion Vectors of Vection-Induced Image and Multivariate Biosignals under Visual Tasks ， BSI2002, Como, Italy ， ， ， June 27, 2002 ，
Correlation between LMV & Biosignals Biosignals Correlation between LMV & middle-range distant-range Right/Left LMV Up/down LMV Correlation coefficient Correlation coefficient 1 1 BP BP 相 関 z 2 z 2 0 0 係 数 -1 -1 30 30 motion[pixel] motion[pixel] 0 0 -30 -30 20 30 40 50 60 70 20 30 40 50 60 70 time[sec] time[sec] No-correlation No-correlation Significant Correlation Relationship between Motion Vectors of Vection-Induced Image and Multivariate Biosignals under Visual Tasks ， BSI2002, Como, Italy ， ， June 27, 2002 ， ，
Influences of Motion Vectors Influences of Motion Vectors Local Motion Vectors （ distant/near ） Global Motion Vectors （ zoom/pan/tilt ） Right/left Up/down Subject BP Z 2 RRInt. BP Z 2 RRInt. BP Z 2 RRInt. ○ / ○ ○ / × ○ / × △ / × × / × × / × ○ / ○ / ○ ○ / ○ / ○ ○ / ○ / ○ A elderly subjects × / ○ △ / ○ ○ / ○ × / × △ / × ○ / × ○ / ○ / ○ ○ / ○ / ○ ○ / ○ / ○ B with mild △ / × ○ / △ △ / ○ ○ / × ○ / △ × / △ × / ○ / ○ ○ / ○ / ○ ○ / ○ / ○ hypertension or C diabetes mellitus D,E,F NA NA NA NA NA NA NA NA NA △ / △ ○ / △ × / × △ / ○ ○ / ○ × / △ △ / △ / △ ○ / ○ / ○ ○ / × / ○ O △ / ○ △ / × ○ / △ △ / ○ △ / ○ ○ / ○ ○ / ○ / △ △ / ○ / △ ○ / ○ / △ P × / △ △ / ○ △ / × △ / ○ △ / ○ × / × △ / × / ○ ○ / △ / ○ ○ / × / △ healthy elderly Q subjects ○ / × × / △ × / △ ○ / ○ △ / ○ △ / ○ △ / ○ / ○ ○ / △ / ○ △ / △ / ○ R × / × △ / △ × / △ △ / ○ ○ / ○ △ / △ × / × / ○ △ / △ / ○ ○ / △ / ○ S Relationship between Motion Vectors of Vection-Induced Image and Multivariate Biosignals under Visual Tasks ， BSI2002, Como, Italy ， ， ， June 27, 2002 ，
Recommend
More recommend
