How Do We Measure Depth Perception in Near-Field Augmented Reality Inspired by Medical Applications? Dr. J. Edward Swan II 25 June 2014 Professor Dept of Computer Science & Engineering Adjunct Professor Dept of Psychology
Augmented Reality (AR) • AR demo from my lab • Virtual objects • At real-world locations • Spatially related to real-world objects
AR Medical Applications and Perceived Depth • AR-based training system for planning brain tumor resection – Robarts Research Institute, Western University, Ontario, Canada – Kamyar Abhari, Terry Peters, and many collaborators • What is the perceived depth of the tumor?
How does depth perception operate? • We see 3D world from ambiguous 2D retinal images • Important scientific question since mid-1800’s • Current theories emphasize importance of depth cues
Near-Field Depth Cues n Occlusion n Relative Size n Relative Density n Binocular Disparity n Accommodation n Convergence n Motion Perspective
Near-Field Depth Cues n Occlusion n Relative Size n Relative Density n Binocular Disparity n Accommodation n Convergence n Motion Perspective
Near-Field Depth Cues n Occlusion n Relative Size n Relative Density n Binocular Disparity n Accommodation n Convergence n Motion Perspective
Near-Field Depth Cues n Occlusion n Relative Size n Relative Density n Binocular Disparity n Accommodation n Convergence n Motion Perspective
Near-Field Depth Cues n Occlusion n Relative Size n Relative Density n Binocular Disparity n Accommodation n Convergence n Motion Perspective
Near-Field Depth Cues n Occlusion n Relative Size n Relative Density n Binocular Disparity n Accommodation n Convergence n Motion Perspective
Near-Field Depth Cues n Occlusion n Relative Size n Relative Density n Binocular Disparity n Accommodation n Convergence n Motion Perspective
Near-Field Depth Cues n Occlusion n Relative Size n Relative Density n Binocular Disparity n Accommodation n Convergence n Motion Perspective
How Do We Measure Depth Perception? • Problem: That’s about a meter… perception is invisible • Solution: depth judgment tasks • Based on cognition – Verbal report • Based on perception-action Who cares? – Perceptual matching Let me hit the ball! – Blind reaching
Previous work in near-field AR depth perception • Rolland et al. [1995] • Rolland et al. [2002] • Ellis & Menges [1998] • McCandless, Ellis, Adelstein [2000] • Edwards et al. [2004] • Singh, Ellis, Swan [2010, 2013] • Hua, Ellis, Swan [2014]
Exp I: Matching vs. Reaching • Used perceptual matching and blind reaching tasks • Extend Ellis & Menges [1998] apparatus and task • Effect of occluding surface (x-ray vision) • Within-subjects design
Exp I: Tasks occluder = absent, present calibration display cross judgment = match judgment = blind reach hydraulic jack
Exp I: Apparatus
Exp I: Results Experiment I 2 1 Error (cm), +/– 1 SEM occluder 0 -1 occluder -2 -3 -4 -5 -6 -7 Distance (cm) 34 38 42 46 50 34 38 42 46 50 34 38 42 46 50 34 38 42 46 50 Occluder absent present absent present Judgment match reach
Exp II Real/AR Matching > Reaching Matching , Reaching Calibration Similar Biomechanics Between-Subjects Exp I Matching/Reaching X-ray Vision Within-Subjects
Exp II: Real vs. AR cameras • Compare real and AR targets background curtain • Compare perceptual matching and blind reaching judgments real-world target • Same biomechanical movement for matching and reaching closed-loop slider foam ridge (restricts HMD movement) • Between-subjects design Environment ¡= ¡R eal , ¡ AR Task ¡= ¡ match Task ¡= ¡ reach
Exp II: Results Experiment II 3 2 1 Error (cm), +/– 1 SEM 0 -1 -2 -3 -4 -5 -6 Distance (cm) 34 38 42 46 50 34 38 42 46 50 34 38 42 46 50 34 38 42 46 50 Environment real AR real AR Judgment match reach
The Accommodative / Vergence Problem for VR and AR vergence ¡distance ¡ ¡ vergence ¡distance ¡ focal ¡distance focal ¡distance ¡ vergence ¡distance ¡ ¡ focal ¡distance Normal viewing Viewing with vergence Viewing with farther than accommodation accommodation farther than vergence
Exp II: Disparity Changes for AR Matching Experiment II 3 2 Error (cm), +/– 1 SEM 1 β 0 -1 Δ v = α – β -2 -3 -4 -5 -6 Distance (cm) 34 38 42 46 50 34 38 42 46 50 34 38 42 46 50 34 38 42 46 50 Environment real AR real AR Judgment match reach α Experiment II AR Matching 0.4 β Δ v = α – β 0.2 Δ vergence ° α 0.0 -0.2 -0.4 34 38 42 46 50 Distance (cm)
Exp II: Results Experiment II 3 2 1 Error (cm), +/– 1 SEM 0 -1 -2 -3 -4 -5 -6 Distance (cm) 34 38 42 46 50 34 38 42 46 50 34 38 42 46 50 34 38 42 46 50 Environment real AR real AR Judgment match reach
Mon-Williams & Tresilian [1999], fig 2 ¡ Exp II Real/AR Matching > Reaching Matching , Reaching Calibration Similar Biomechanics Between-Subjects Exp I Matching/Reaching X-ray Vision Within-Subjects Matching > Reaching No Calibration Matching > Reaching Exp III Calibration Feedback Use Finger Matching , Reaching Real, AR Pretest, Intervention, Posttest
Exp III: Feedback, Proprioception • Feedback: – Pretest , Expose , Posttest design – Reach-1, Match-2, Reach-3 – Match-1, Reach-2, Match-3 • Point with Finger: – Reaching and matching both use finger • Compare real and AR targets
Exp III: Tasks blind cardboard reaching ridge perceptual matching
Comparing Exp II Experiment II 5 and Exp III Pretest 4 3 Error (cm), +/– 1 SEM 2 1 0 -1 -2 -3 -4 -5 -6 Distance (cm) 34 38 42 46 50 34 38 42 46 50 34 38 42 46 50 34 38 42 46 50 Environment real AR real AR Judgment match reach Experiment III 5 4 3 Error (cm), +/– 1 SEM 2 1 0 -1 -2 -3 -4 -5 -6 Distance (%) 55 63 71 79 87 55 63 71 79 87 55 63 71 79 87 55 63 71 79 87 Environment real AR real AR Judgment match reach
Exp III: Feedback Experiment III Environment = real (Pretest, 5 4 Error (cm), +/– 1 SEM 3 Expose, 2 1 Posttest) 0 -1 -2 -3 -4 55 63 71 79 87 55 63 71 79 87 55 63 71 79 87 55 63 71 79 87 55 63 71 79 87 55 63 71 79 87 Distance (%) reach-1 match-2 reach-3 match-1 reach-2 match-3 Judgment Block Experiment III Environment = AR 5 4 Error (cm), +/– 1 SEM 3 2 1 0 -1 -2 -3 -4 5 3 1 9 7 5 3 1 9 7 5 3 1 9 7 5 3 1 9 7 5 3 1 9 7 5 3 1 9 7 Distance (%) 5 6 7 7 8 5 6 7 7 8 5 6 7 7 8 5 6 7 7 8 5 6 7 7 8 5 6 7 7 8 1 2 3 1 2 3 Judgment Block - - - - - - h h h h h h c c c c c c a t a t a t a a a e e e m m m r r r
Exp III: Posttest Results (Reach-3, Match-3) Experiment III (Posttest) 5 4 3 Error (cm), +/– 1 SEM 2 1 0 -1 -2 -3 -4 -5 -6 Distance (%) 55 63 71 79 87 55 63 71 79 87 55 63 71 79 87 55 63 71 79 87 Environment real AR real AR Judgment match reach
Exp II Real/AR Matching > Reaching Matching , Reaching Calibration Similar Biomechanics Between-Subjects Overestimation AR-Matching Exp IV Exp I Overestimation Accommodation AR-Matching Matching/Reaching Collimated, Consistent, X-ray Vision Midpoint, Real Within-Subjects Younger Matching > Reaching Between-Subjects No Calibration Overestimation AR-Matching Matching > Reaching Exp III Calibration Feedback Use Finger Matching , Reaching Real, AR Pretest, Intervention, Posttest
Haploscopes • Commercial HMDs don’t allow enough control • Vision scientists have long built haploscopes : Presents separately- controlled image to each eye Berkeley Haploscope HASA Haploscope (HMD) Clemson Haploscope 32
Haploscope Monitor Minimization Lens Collimation Lens Accommodation Lens (Concave) (Convex) (Concave) 10 cm Stimulus Object -10 D +10 D § Adjustable focus display § Can present objects with varying accommodation 5 cm Image generated by Image is formed at 33.3 - 50 cm minimization lens 10 cm and vergence demands based on the power of the accommodation lens f Image generator m n Optics Eye positions as pivot points
Exp IV: Accommodation • Environment / Accommodation: – Real / Consistent – AR / Collimated – AR / Consistent – AR / Midpoint Environment ¡= ¡R eal, ¡AR Haploscope • Perceptual Matching Task ¡= ¡ match • 33 to 50 cm
Exp IV: Results Experiment IV 2.5 2.0 Error (cm), +/– 1 SEM 1.5 1.0 0.5 0.0 -0.5 -1.0 3 4 0 4 0 3 4 0 4 0 3 4 0 4 0 3 4 0 4 0 Distance (cm) . . . . . . . . . . . . . . . . . . . . 3 6 0 4 0 3 6 0 4 0 3 6 0 4 0 3 6 0 4 0 3 3 4 4 5 3 3 4 4 5 3 3 4 4 5 3 3 4 4 5 l d t t a Condition n n e e e i t o R a t p s m i d s i i n l M l o o C C
Recommend
More recommend
