Time-Varying Behavior of Motion Vectors in Vection-Induced Images - PowerPoint PPT Presentation

Time-Varying Behavior of Motion Vectors in Vection-Induced Images In Relation to Autonomic Regulation Tohru Kiryu 1 , Hiroshi Yamada 1 , Masahiro Jimbo 1 , and Takehiko Bando 2 1 Graduate School of Science and Technology, 2 Graduate School of Medical and Dental Sciences, Niigata University, Niigata, Japan Abstract — Virtual reality (VR) is a promising technology in biomedical engineering, but at the same time enlarges another problem called cybersickness. Aiming at suppression of cybersicknes, we are investigating the influences of vection-induced images on the autonomic regulation quantitatively. We used the motion vectors to quantify image scenes and measured electrocardiogram, blood pressure, and respiration for evaluating the autonomic regulation. Using the estimated motion vectors, we further synthesized random-dot pattern images to survey which component of the global motion vectors seriously affected the autonomic regulation. The results showed that the zoom component with a specific frequency band (0.1 – 3.0 Hz) would induce sickness. Keywords — cybersickness, autonomic regulation, motion vector, vection-induced image, random-dot pattern

Background -Development of digital imaging technology is producing many image formats, resolutions, frame rates, in addition to conventional factors. -Current digital imaging technology is also creating extraordinary special effects that we have never seen or experienced. - Contrary to the benefits, digital imaging technology is widely spreading unexpected visual stimulus. -Not only entertainment, but also practical problems are emerging especially in the virtual reality (VR) or the virtual environment (VE). -Regarding visually induced illusions of self-motion, it has been reported that the mismatch between visual system and vestibular system causes sickness (sensory conflict theory) . Cybersickness utonomic Regulation � IEEE EMBS04 at San Francisco � September 4, 2004 Time-Varying Behavior of Motion Vectors in Vection-Induced Images In Relation to A

Approaches 1. Database of Biosignals under Vection-Induced Images 2. Quantifying the Image Components by Motion Vectors 3. Featuring Motion Vectors around Sickness Intervals that were Determined by Biosignals. 4. Estimation of System Function by Multivariate ARX Model 0.1-3.0 Hz band power of zoom component 4 Band power of zoom component 2 0 motion of zoom component 0.2 zoom component 0 -0.2 HF and LF power from R-R interval time-series 2 RR LF 1.5 HF & LF power of R-R interval RR HF 1 0.5 0 0 50 120 100 time [sec] utonomic Regulation � IEEE EMBS04 at San Francisco � September 4, 2004 Time-Varying Behavior of Motion Vectors in Vection-Induced Images In Relation to A

Overview watching biosignal biosignal extraction extraction motion vector motion vector real image real image synthesis synthesis biosignal biosignal watching random dot pattern by CG random dot pattern by CG utonomic Regulation � IEEE EMBS04 at San Francisco � September 4, 2004 Time-Varying Behavior of Motion Vectors in Vection-Induced Images In Relation to A

Experiments under Real & RDP Images Experimental Protocol Experimental Protocol real images real images Vehicle experiencing video ... ... real images rest rest real 3 min 3 min about 18 min Parachute task # 3 5 7 rest s s pt s zpt s zt s zpt s zp s zpt s rest RDP Bobsleigh 3 min 3 min about 18 min boat boat s: still, z: zoom, p: pan, t: tilt Go cart Hang glider Subjects Subjects Mountain-bike ten healthy young subjects (eight males and two Mountain-bike female from 21 to 24 yrs. old) Car race Car race Bungee jump diving Measured Biosignals Measured Biosignals diving Bike race ECG : chest Bike race Respiration : tube sensors around the chest and the abdomen Blood Pressure : tonometry method at Niigata University � December, 2002 � utonomic Regulation � IEEE EMBS04 at San Francisco � September 4, 2004 Time-Varying Behavior of Motion Vectors in Vection-Induced Images In Relation to A

Quantification of Image by Motion Vectors Local Motion Vector Local Motion Vector Global Motion Vector Global Motion Vector pan zoom distant view camera tilt local motion in a screen local motion in a screen motion of camera motion of camera post frame y current frame y tilt Bottom up approach x x Block matching method utonomic Regulation � IEEE EMBS04 at San Francisco � September 4, 2004 Time-Varying Behavior of Motion Vectors in Vection-Induced Images In Relation to A

Correlation between Pan and Right/Left (1,1) (1,2) (1,3) (1,4) (1,5) (2,1) (2,2) (2,3) (2,4) (2,5) (3,1) (3,2) (3,3) (3,4) (3,5) (4,1) (4,2) (4,3) (4,4) (4,5) (5,1) (5,2) (5,3) (5,4) (5,5) | correlation coefficient | � 0.7 mountain-bike mountain-bike time [sec] utonomic Regulation � IEEE EMBS04 at San Francisco � September 4, 2004 Time-Varying Behavior of Motion Vectors in Vection-Induced Images In Relation to A

Correlation between Pan and Right/Left (1,1) (1,2) (1,3) (1,4) (1,5) (2,1) (2,2) (2,3) (2,4) (2,5) (3,1) (3,2) (3,3) (3,4) (3,5) (4,1) (4,2) (4,3) (4,4) (4,5) (5,1) (5,2) (5,3) (5,4) (5,5) bobsleigh | correlation coefficient | � 0.7 time [sec] utonomic Regulation � IEEE EMBS04 at San Francisco � September 4, 2004 Time-Varying Behavior of Motion Vectors in Vection-Induced Images In Relation to A

GMV and Time-Frequency Representation [magnification] zoom Frequency [Hz] pan [pixel] Frequency [Hz] tilt [pixel] Frequency [Hz] time [sec] mountain-bike mountain-bike utonomic Regulation � IEEE EMBS04 at San Francisco � September 4, 2004 Time-Varying Behavior of Motion Vectors in Vection-Induced Images In Relation to A

Sickness Interval and Trigger Point Estimation of Trigger Points RES � � HF HF RES HF RR (Respiration, HRV) HF RR (Respiration, HRV) � LF � HRV) BP � (Blood Pressure � LF BP LF LF RR RR (Blood Pressure HRV) 1.8 RES � � HF HF RES HF RR Signal Intensity HF RR 1.4 1.0 Normalized data by values during 3-min rest 0.8 0.8 0.6 Sickness Interval Z Z s 0.2 s 0 20 40 60 80 100 120 Time [sec] � ( HF � LF ) � LF RES _ 80 % BP _ 120 % BP _ 120 % 1.8 � LF BP � LF BP LF RR LF Signal Intensity RR 1.4 ( HF LF ) LF � � � 1.2 1.2 RR _ 80 % RR _ 120 % RR _ 120 % 1.0 0.6 Trigger Point � t t � � 0.2 0 20 40 60 80 100 120 - local minimum of LF LF BP time [sec] t � t Z s Z BP � s - local minimum of LF LF RR RR utonomic Regulation � IEEE EMBS04 at San Francisco � September 4, 2004 Time-Varying Behavior of Motion Vectors in Vection-Induced Images In Relation to A

Distribution of Trigger Points Trigger Points t g as a function of time - real image (mountain bike) - random dot pattern image t g t g 0 20 40 60 80 100 120 0 20 40 60 80 100 120 Time [sec] Time [sec] 4 8 N=52 N=19 3 6 Repetition Repetition 2 4 1 2 0 0 t1 t2 t3 t4 t5 t6 t7 t8 t9 t10 t11 t12 t1 t2 t3 t4 t5 t6 t7 t8 t9 t10 t11 t12 Segment Segment t g g = 41, 49, 93, 99 [sec] t g g = 50, 59, 103, 110 [sec] t t t1:1-10 [sec], t2:11-20 [sec], t3:21-30 [sec], t4:31-40 [sec], t5:41-50 [sec] t5:41-50 [sec] , t6:51-60 [sec] t6:51-60 [sec] , t7:61-70 [sec], t8:71-80 [sec], t9:81-90 [sec], t10:91-100 [sec] t10:91-100 [sec] , t11:101-110 [sec] t11:101-110 [sec] , t12:111-120 [sec] utonomic Regulation � IEEE EMBS04 at San Francisco � September 4, 2004 Time-Varying Behavior of Motion Vectors in Vection-Induced Images In Relation to A

Time-Frequency Structure of GMVs at Trigger Points zoom zoom 15 15 10 10 5 5 frequency [ Hz ] frequency [ Hz ] 0 0 pan pan 15 15 10 10 5 5 0 0 tilt tilt 15 15 10 10 5 5 0 0 35 37 39 41 43 45 47 41 43 45 47 49 49 51 53 55 time [ sec ] t g t time [ sec ] t g t g g zoom zoom 15 15 10 10 5 5 frequency [ Hz ] 0 frequency [ Hz ] 0 pan pan 15 15 10 10 5 5 0 0 tilt 15 tilt 15 10 10 5 5 0 0 87 89 91 93 93 95 97 99 93 95 97 99 99 101 103 105 time [ sec ] t g t time [ sec ] t t g g g utonomic Regulation � IEEE EMBS04 at San Francisco � September 4, 2004 Time-Varying Behavior of Motion Vectors in Vection-Induced Images In Relation to A

Extraction of Trigger GMV by Similarity Survey for Similar Time-Frequency Structure MV( t ): GMV ( zoom, pan, tilt ) k j-2 Signal Intensity Wavelet Transform P( t ) k j-1 Power of MV k j ��� n 2 2 � P ( t ) (Re{ MV ( f , t )}) (Im{ MV ( f , t )}) = + i i time t = 1, 2 ��� ( j-1) j ( j+1) ��� i 1 = f : frequency, t : time - frequency band: 0.01- 15 [Hz] �� � n. of bands, n = 31 � V 0 = ( mP zoom , mP pan , mP tilt ) -mP( t ): mean of Power V ( t ) = {mP zoom ( t ), mP pan ( t ), mP tilt ( t ) } for each section (k 1 , k 2 , ��� , k j , ��� ) � � 2 ( v 0 � v ) � � � ( t ) = cos Similarity Similarity � � - interval: 3 [sec] � 90 [point] � � � v 0 � v � � - shift: 1 [sec] � 30 [point] � � � utonomic Regulation � IEEE EMBS04 at San Francisco � September 4, 2004 Time-Varying Behavior of Motion Vectors in Vection-Induced Images In Relation to A