DiCE Dichoptic Contrast Enhancement for VR and Stereo Displays Fangcheng Zhong University of Cambridge George Alex Koulieris Durham University & Université Côte d’Azur Inria George Drettakis Université Côte d’Azur Inria Martin S. Banks UC Berkeley Mathieu Chambe ENS Rennes Fredo Durand MIT Rafał K. Mantiuk University of Cambridge
Why is contrast important? High contrast Low contrast • Colour richness • Realism • 3D appearance • Details 2
How to show high contrast in VR HDR display Local tone-mapping operators • Cost • Computational cost • Power • Flicker 3
Exploiting binocular vision Idea • Use different views between the eyes to enhance image appearance • *Need to take care of Binocular Rivalry Related work • Binocular tone-mapping operators (BTMO): [Yang et al. 2012] [Zhang et al. 2018, 2019] • Maximize image difference, yet maintaining viewing comfort Problems • Inconsistent enhancement • Heuristic viewing comfort predictor • Heavy optimization Left-eye image Right-eye image 4
DiCE: Dichoptic* Contrast Enhancement Left-eye Image Right-eye Image *Dichoptic presentation: two different images are presented to the two eyes 5
Tone-curve and logarithmic contrast log(𝑀 max ) Slope < 1 Tone-curve a mapping from Contrast reduced the luminance of the input image to the luminance of Slope > 1 display device Slope = 1 𝑚𝑝𝑏𝑠𝑗𝑢ℎ𝑛𝑗𝑑 𝑑𝑝𝑜𝑢𝑠𝑏𝑡𝑢 Contrast preserved Contrast boosted = log(𝑀 max ) − log(𝑀 min ) 𝑈ℎ𝑓 𝑡𝑚𝑝𝑞𝑓 𝑏𝑒𝑘𝑣𝑡𝑢𝑡 𝑑𝑝𝑜𝑢𝑠𝑏𝑡𝑢 log(𝑀 min ) Cannot increase the contrast globally without a larger output dynamic range 6
Fusion of contrast Contrast to the right eye Contrast to the left eye 1 𝛾 + 𝐷 𝑆 𝛾 𝛾 𝐷 𝑀 𝐷 𝑀 𝐷 𝑆 𝐷 𝑛 = 2 𝛾 ≈ 3 Perceived contrast 𝐷 𝑛 ≥ 𝐷 𝑀 + 𝐷 𝑆 ⟹ 2 [Legge & Rubin 1981] 7
Dichoptic tone-curves Left-eye tone-curve Right-eye tone-curve S 𝑚 S 𝑠 Perceived tone-curve 8
Predictor of rivalry Left-eye tone-curve Right-eye tone-curve Need a predictor of rivalry! Predictor hypothesis 1 • Luminance difference d 2𝑒 h Predictor hypothesis 2 d • Ratio of contrast 𝑚 l ℎ 9
Experiment 1 - Predictor of rivalry Stimuli • 16 binocular images processed by Right-eye tone-curve interleaved tone-curves Left-eye tone-curve Procedure • Adjust the difference d to find the strongest enhancement without rivalry (8 participants) • Run the experiment for different number of linear segment N N=14 N=2 Which predictor is closer to a constant for different N? Interleaved tone-curve Hypothesis 1 Hypothesis 2 Ratio of contrast Luminance difference 𝑚 2𝑒 10 ℎ
Experiment 1 - Predictor of rivalry Hypothesis 1 Hypothesis 2 Luminance difference Ratio of contrast 𝑚 2𝑒 ℎ Better! 11
Selected examples N=2 l/h=0.33 h-l=0.36 N=2 l/h=0.33 h-l=0.36 The two image pairs have the same ratio of contrast but different N=20 l/h=0.33 h-l=0.136 N=20 l/h=0.33 h-l=0.136 luminance difference These images can be cross-fused with the assistance of the dots above 12
Final DiCE tone-curve Output log luminance 0 • The shape of the DiCE dichoptic -0.5 tone-curves for different ratios of contrast l/h -1 l/h =0.3 3289, 1th pctl -1.5 • Select the ratio that represents l/h =0.5287, 25th pctl the 50 th percentile of the data l/h =0.6308, 50th pctl -2 l/h =0.7422, 75th pctl to shape the final tone-curve -2.5 l/h =1.0000, 99th pctl -2.5 -2 -1.5 -1 -0.5 0 Intput log luminance 13
Integration of DiCE to the VR rendering pipeline • Seamless integration to any VR rendering pipeline Standard VR • Can take both HDR and SDR images as input rendering pipeline • Negligible computational cost 14
Experiment 2 – Validation Pairwise Comparison DiCE Contrast Preference BTMO Standard Images without enhancement binocular tone-mapping Run comparisons for 19 scenes (16 participants) 15
Experiment 2 – Validation (Contrast Perception) Fix the quality of standard images to zero Plot the quality scores of DiCE / BTMO relatively 16
Experiment 2 – Validation (Overall Preference) Fix the quality of standard images to zero Plot the quality scores of DiCE / BTMO relatively 17
Summary Contributions: • Explicit contrast enhancement based on established psychophysical models. • A simple yet effective rivalry predictor based on new experimental findings. • Easy integration to any VR rendering pipeline with any choice of tone-mapping operators. • Negligible computational cost, processing an image pair in milliseconds without GPU acceleration. Limitations: • Inherent trade-off between contrast enhancement and binocular rivalry. 18
19 THANK YOU! Project URL (Unity Asset available) https://www.cl.cam.ac.uk/research/rainbow/projects/dice/ QR code DiCE demo available in the session break
Recommend
More recommend
