Computational Focus-Tunable Near-eye Displays Nitish Padmanaban - PowerPoint PPT Presentation

Computational Focus-Tunable Near-eye Displays � Nitish Padmanaban � Stanford University � NVIDIA GPU Technology Conference 2017 � www.computationalimaging.org �

Magnified Display � d � d’ � f � 1 d + 1 d ' = 1 f

Real World: � Vergence & � Accommodation � Match! Match! �

� Current VR Displays: � Vergence & � Accommodation � Mismatch Mismatch � for people � with normal vision �

Nearsightedness & Farsightedness � Focal range (range of clear vision) � Normal vision � Nearsighted/myopic � Farsighted/Hyperopic � Presbyopic � Optical Infinity � 4D / 25cm � Modified from Pamplona et al, Proc. of SIGGRAPH 2010 �

Presbyopia � Nearest focus distance (D) � 16D/6cm � 12D/8cm � 8/12.5cm � 4/25cm � 0/ ∞� 8 � 16 � 24 � 32 � 40 � 48 � 56 � 64 � 72 � Age (years) � Duane, 1912 �

• Q1: Can computational displays effectively replace glasses in VR/AR? � • Q2: How to address the vergence–accommodation conflict for users of different ages? � • Q3: What are some near-eye display technologies that address this? �

• Q1: Can computational displays effectively replace glasses in VR/AR? � • Q2: How to address the vergence–accommodation conflict for users of different ages? � • Q3: What are some near-eye display technologies that address this? �

Fixed Focus � Lens � f � d’ � d � 1 d + 1 d ' = 1 Display � Magnified Display � f

Adaptive Focus � actuator à vary d’ � Lens � 1 d + 1 d ' = 1 Display � Magnified Display � f

Adaptive Focus � focus-tunable � lens à vary f � Lens � 1 d + 1 d ' = 1 Display � Magnified Display � f

Padmanaban et al., PNAS 2017 �

Padmanaban et al., PNAS 2017 �

Task � 3D � 2D � 1D � 4D � (0.33m) � (0.50m) � (1m) � (0.25m) � Four simulated distances Four simulated distances � How sharp is the target? (blurry, medium, sharp) � Is the target fused? (yes, no) �

Results: Sharpness and Fusibility � 1D � 2D � 3D � 4D � 1D � 2D � 3D � 4D � far 1m � 0.5m � 0.3m � 0.25m � 1m � 0.5m � 0.3m � 0.25m � far near far near Distance � Distance �

Results: Sharpness and Fusibility � 1D � 2D � 3D � 4D � 1D � 2D � 3D � 4D � far 1m � 0.5m � 0.3m � 0.25m � 1m � 0.5m � 0.3m � 0.25m � far near far near Distance � Distance �

Results: Sharpness and Fusibility � 1D � 2D � 3D � 4D � 1D � 2D � 3D � 4D � far 1m � 0.5m � 0.3m � 0.25m � 1m � 0.5m � 0.3m � 0.25m � far near far near Distance � Distance �

Results: Sharpness and Fusibility � (n = 64) 1D � 2D � 3D � 4D � 1D � 2D � 3D � 4D � far far 1m � 0.5m � 0.3m � 0.25m � 1m � 0.5m � 0.3m � 0.25m � far near far near Distance � Distance �

Summary � • Fast, user-driven refractive estimates can be used to correct for near and far sightedness in an AR/VR system so that the user does not need to wear their typical correction �

Computational Near-eye Displays � • Q1: Can computational displays effectively replace glasses in VR/AR? � • Q2: How to address the vergence–accommodation conflict for users of different ages? � • Q3: What are some near-eye display technologies that address this? �

Conventional Stereo / VR Display � Conventional stereoscopic virtual image distance distance stereoscopic distance vergence � accommodation �

Removing VAC with Adaptive Focus � With Focus Cues virtual image distance stereoscopic distance stereoscopic distance vergence � accommodation �

Task � 0.5D � 4D � (2m) � (0.25m) � Follow the target with your eyes �

Accommodative response in conventional display � Conventional stereoscopic virtual image distance distance Stimulus stereoscopic distance Padmanaban et al., PNAS 2017 �

Accommodative response in conventional display � Conventional stereoscopic virtual image distance distance Stimulus stereoscopic Accommodation distance n = 59, mean gain = 0.29 � Padmanaban et al., PNAS 2017 �

Accommodative response in focus tunable display � With Focus Cues virtual image distance stereoscopic distance Stimulus stereoscopic distance Padmanaban et al., PNAS 2017 �

Accommodative response in focus tunable display � With Focus Cues virtual image distance stereoscopic distance Stimulus Accommodation stereoscopic distance n = 24, mean gain = 0.77 � Padmanaban et al., PNAS 2017 �

Presbyopia � Focal range (range of clear vision) � Normal vision � Nearsighted/myopic � Farsighted/Hyperopic � Presbyopic � Optical Infinity � 4D / 25cm � Modified from Pamplona et al, Proc. of SIGGRAPH 2010 �

Do Presbyopes benefit from dynamic focus? � Padmanaban et al., PNAS 2017 �

Do Presbyopes benefit from dynamic focus? � Padmanaban et al., PNAS 2017 �

Do Presbyopes benefit from dynamic focus? � Padmanaban et al., PNAS 2017 �

Image quality in VR � far near far near Padmanaban et al., PNAS 2017 �

Image quality in VR � far near far near Padmanaban et al., PNAS 2017 �

Image quality in VR � far near far near Padmanaban et al., PNAS 2017 �

Image quality in VR � far near far near Padmanaban et al., PNAS 2017 �

Image quality in VR � far near far near Padmanaban et al., PNAS 2017 �

Image quality in VR � far near far near Padmanaban et al., PNAS 2017 �

Summary � • The best approach for mitigating the vergence– accommodation conflict may differ depending on the age of the user � • For users over the age of 45, a “conventional” stereo display may actually provide better image quality, particularly for nearby virtual objects. � • However, somewhat paradoxically, the dynamic display did improve fusion for all ages �

• Q1: Can computational displays effectively replace glasses in VR/AR? � • Q2: How to address the vergence–accommodation conflict for users of different ages? � • Q3: What are some near-eye display technologies that address this? �

• Q1: Can computational displays effectively replace glasses in VR/AR? � • Q2: How to address the vergence–accommodation conflict for users of different ages? � • Q3: What are some near-eye display technologies that address this? � • Gaze-contingent focus � • Monovision � • Light field displays � • etc �

Gaze-contingent Focus � • non-presbyopes: adaptive focus is like real world, but needs eye tracking! � virtual image � HMD � micro display � lens � eye tracking � Padmanaban et al., PNAS 2017 �

Gaze-contingent Focus � Padmanaban et al., PNAS 2017 �

Gaze-contingent Focus � Padmanaban et al., PNAS 2017 �

Gaze-contingent Focus � Padmanaban et al., PNAS 2017 �

Gaze-contingent Focus – User Preference � Padmanaban et al., PNAS 2017 �

Monovision VR � Konrad et al., SIGCHI 2016; Johnson et al., Optics Express 2016; Padmanaban et al., PNAS 2017 �

� Monovision VR � • monovision did not drive accommodation more than conventional � • visually comfortable for most; particularly uncomfortable for some users � Konrad et al., SIGCHI 2016; Johnson et al., Optics Express 2016; Padmanaban et al., PNAS 2017 �

Light Field Stereoscope � Thin Spacer & 2 nd panel (6mm) � Backlight � LCD Panel � Magnifying Lenses � Huang et al., SIGGRAPH 2015 �

Light Field Stereoscope � Light Field Cameras � Huang et al., SIGGRAPH 2015 �

� � � � � Summary � • focus cues in VR/AR are challenging � • adaptive focus can correct for refractive errors (myopia, hyperopia) � • gaze-contingent focus gives natural focus cues for non-presbyopes, but require eyes tracking � • presbyopes require fixed focal plane with correction � • monovision has not demonstrated significant improvements � • light field displays may be the “ultimate” display, but need special considerations for presbyopes �

� � � � � Making Virtual Reality Better Than Reality? � • focus cues in VR/AR are challenging � • adaptive focus can correct for refractive errors (myopia, hyperopia) � • gaze-contingent focus gives natural focus cues for non-presbyopes, but require eyes tracking � • presbyopes require fixed focal plane with correction, better than reality! � • monovision has not demonstrated significant improvements � • light field displays may be the “ultimate” display, but need special considerations for presbyopes �

Acknowledgements � • Gordon Wetzstein (Stanford) � • Robert Konrad (Stanford) � • Fu-Chung Huang (NVIDIA) � • Emily Cooper (Dartmouth College) � • Tal Stramer (Stanford) �