Realistic Image Synthesis - Perception-based Rendering - Philipp Slusallek Karol Myszkowski Gurprit Singh Realistic Image Synthesis SS19 – Perception-based Rendering Karol Myszkowski
Making Rendering Efficient • Realistic image synthesis goal – Generate an image that evokes from the visual perception system a response that is indistinguishable from that evoked by the original environment – Global illumination important component of realism • The solution of the global illumination problem is computationally hard: – Take into account characteristics of the Human Visual System to concentrate the computation exclusively on the visible scene details Realistic Image Synthesis SS19 – Perception-based Rendering
Outline • Perceptually based adaptive sampling algorithm • Steering Monte Carlo ray (path) tracing using perception inspired image quality metrics • Image-based rendering for animations • Eye tracking driven rendering Realistic Image Synthesis SS19 – Perception-based Rendering
A Perceptually Based Adaptive Sampling Algorithm by Mark Bolin & Gary Meyer SIGGRAPH 1998 • Uses a multi-scale visual model (the Sarnoff Visual Discrimination Model) to guide the sampling pattern in MC Ray Tracing – Optimized for speed • Haar wavelets are used at the cortex filtering stage instead of costly Laplacian pyramid originally used in the VDM – Correct color handling • CIE XYZ transformed to SML space modeling retinal cone sensitivity • Opponent contrast space: a single achromatic (A) and two opponent color channels (C 1 and C 2 ) • Independent contrast sensitivity processing for AC 1 C 2 channels Realistic Image Synthesis SS19 – Perception-based Rendering
Chromatic CSF Independent contrast sensitivity processing for AC 1 C 2 channels Band-pass filter Low-pass filter Blue-Yellow Opponent Luminance Red-Green Opponent Realistic Image Synthesis SS19 – Perception-based Rendering
Visual Masking • Achromatic and chromatic CSFs with noise (left), and perceptual metric response in the comparison with noiseless CSF (right). • Brighter shades denote better noise visibility (less masking). Realistic Image Synthesis SS19 – Perception-based Rendering
Visual Masking A chapel image without (left) and with imposed sinusoidal distortion (center). Visual difference metric results (right): brighter shades of grey denote less masking and better visibility of the sinusoidal distortion pattern. Realistic Image Synthesis SS19 – Perception-based Rendering
Perception-based Adaptive Sampling • Step I: compute an estimate of the image using lesser number of samples per pixel – A Haar wavelet image approximation is generated and then refined • Step II: from MC variance in samples of each pixel estimate the pixel error bounds. – The error expressed in terms of the variance of the detail terms in the Haar image representation Realistic Image Synthesis SS19 – Perception-based Rendering
Perception-based Adaptive Sampling • Step III: from an Estimated Image and error-bounds compute a Lower Bound Image and an Upper Bound Image. Realistic Image Synthesis SS19 – Perception-based Rendering
Perception-based Adaptive Sampling • Step IV: Compute oriented band-pass images. Realistic Image Synthesis SS19 – Perception-based Rendering
Perception-based Adaptive Sampling • Step V: For each band compute threshold from TVI, CSF and Masking functions. Normalize the band pass images with the computed threshold. Realistic Image Synthesis SS19 – Perception-based Rendering
Perception-based Adaptive Sampling • Step VI: Find the difference between each band of the two images. • Step VII : Refine the area with maximum difference. Realistic Image Synthesis SS19 – Perception-based Rendering
Perception-based Adaptive Sampling • Algorithm summary Realistic Image Synthesis SS19 – Perception-based Rendering
Perception-based Adaptive Sampling Image Sample Density Realistic Image Synthesis SS19 – Perception-based Rendering
A Perceptually Based Physical Error Metric for Realistic Image Synthesis by Mahesh Ramasubramanian, Sumanta N. Pattanaik, and Donald P. Greenberg Siggraph 1999 Aims for perceptual accuracy • Limitations of the human visual system... perceptual accuracy < physical accuracy . • Perceptual accuracy guides rendering, not physical accuracy. Realistic Image Synthesis SS19 – Perception-based Rendering
A Perceptually Based Physical Error Metric for Realistic Image Synthesis by Mahesh Ramasubramanian, Sumanta N. Pattanaik, and Donald P. Greenberg Siggraph 1999 Aims for perceptual accuracy • Limitations of the human visual system... perceptual accuracy < physical accuracy . • Perceptual accuracy guides rendering, not physical accuracy. Realistic Image Synthesis SS19 – Perception-based Rendering
Preview 6% effort effort distribution (darker regions - less effort) physically perceptually accurate accurate Realistic Image Synthesis SS19 – Perception-based Rendering
Preview 6% effort effort distribution (darker regions - less effort) physically perceptually accurate accurate Realistic Image Synthesis SS19 – Perception-based Rendering
Perceptually Based Rendering Traditional approach: Pair of images to compare at each time step start render (a) intermediate images at consecutive time steps. perceptual error (b) upper and lower bound good n images at each time step. enough ? y done Realistic Image Synthesis SS19 – Perception-based Rendering
Perceptual Error Metric Vision model - expensive physical domain perceptual domain visual vision model rep. 1 < perceptual = threshold perceptual visual vision model difference rep. 2 Realistic Image Synthesis SS19 – Perception-based Rendering
Perceptually Based Physical Error Metric physical domain perceptual domain < < physical perceptual = threshold threshold perceptual difference Realistic Image Synthesis SS19 – Perception-based Rendering
Physical Threshold Map Predicted bounds of permissible luminance error 25% 4% threshold model 30% input image physical threshold (brighter regions - higher thresholds) Realistic Image Synthesis SS19 – Perception-based Rendering
Threshold Model Components luminance frequency contrast image threshold component component component map Realistic Image Synthesis SS19 – Perception-based Rendering
Threshold Model 1. Luminance component 4 TVI log threshold 2 0 2% -2 -4 -2 0 2 4 threshold due to luminance log adaptation luminance Realistic Image Synthesis SS19 – Perception-based Rendering
Threshold Model 2. Frequency component 2% log threshold factor inverse 100 CSF 10 15% 1 .1 1 10 threshold due to luminance + freq. log Spatial Frequency (cpd) Realistic Image Synthesis SS19 – Perception-based Rendering
Threshold Model 3. Contrast component (visual masking) 15% log threshold factor masking function 30% threshold due to luminance + freq. log contrast + contrast Realistic Image Synthesis SS19 – Perception-based Rendering
Validation + = image noise image + noise Realistic Image Synthesis SS19 – Perception-based Rendering
Threshold Model luminance frequency contrast image threshold component component component map Realistic Image Synthesis SS19 – Perception-based Rendering
Global Illumination Revisited = + global direct indirect illumination illumination illumination (fast) (slow) Realistic Image Synthesis SS19 – Perception-based Rendering
Threshold Model Revisited spatially-dependent processing 1 time 12 sec precompute direct illum. luminance-dependent processing N times 0.1 s iterate partial global illum. Realistic Image Synthesis SS19 – Perception-based Rendering
Adaptive Rendering Algorithm start spatial direct precompute info. illumination refine global illumination perceptual iterate error good n enough ? y done Realistic Image Synthesis SS19 – Perception-based Rendering
Results 5% effort effort distribution (darker regions - less effort) reference adaptive solution solution Realistic Image Synthesis SS19 – Perception-based Rendering
Results: Masking by Textures 5% effort effort distribution (darker regions - less effort) reference adaptive solution solution Realistic Image Synthesis SS19 – Perception-based Rendering
Results 5% effort + = noisy masked adaptive adaptive direct indirect global illumination illumination illumination Realistic Image Synthesis SS19 – Perception-based Rendering
Results: Masking by Geometry 5% effort effort distribution (darker regions - less effort) reference adaptive solution solution Realistic Image Synthesis SS19 – Perception-based Rendering
Results: Masking by Shadows 6% effort effort distribution (darker regions - less effort) reference adaptive solution solution Realistic Image Synthesis SS19 – Perception-based Rendering
Discussion New and efficient perceptually based global illumination technique. Advantage: Exploits spatial information in scene, but computes it only once. Limitation: Only for view-dependent rendering. Incorporating temporal sensitivity. Realistic Image Synthesis SS19 – Perception-based Rendering
Recommend
More recommend
