Probabilistic illumination-aware filtering for Monte Carlo rendering - PowerPoint PPT Presentation

Probabilistic illumination-aware filtering for Monte Carlo rendering Ian C. Doidge Mark W. Jones Swansea University, Wales, UK Wednesday 12 th June Computer Graphics International 2013

Monte Carlo Integration − Path Tracing − Unbiased − Robust 16384 samples � 10 hrs 2

Monte Carlo Integration − Path Tracing − Unbiased − Robust − Progressive − High variance: − Noise − Converges with 4096 samples � 2 hrs 30 mins 2048 samples � 1 hr 20 mins 64 samples � 2 mins 30 secs 32 samples � 1 min 10 secs 1024 samples � 40 mins 16384 samples � 10 hrs 256 samples � 10 mins 512 samples � 20 mins 128 samples � 5 mins 16 samples � 34 secs 8192 samples � 5 hrs 8 samples � 17 secs 4 samples � 8 secs 2 samples � 4 secs 1 sample � 2 secs 3

Motivation − Can exploit pixel correlation − But... often breaks down: − Texture detail − Illumination discontinuities − Glossy reflections − Filter but detect & preserve edges − Use data from rendering Path Tracing Reference − Already available

Related Work Outlier rejection: − For ‘cleaning up’ images − Decoro et al. ‘10, Pajot et al. ’11 General image denoising: − Kalantari & Sen ‘13 Sample reconstruction: − Sen & Darabi ’12 − Impressive quality, few samples − Unsuitable for progressive rendering (Ir)radiance caching: − Ward ‘88, Kontkanen ’04, Krivanek ‘05, Gassenbauer ’09 − Interpolation bias, limited to low frequency detail

Contributions − Per-pixel clustering framework − Incident radiance − Layers to separate vertex contributions Path Tracing − Aids detection of discontinuities − High intensity noise removal − Pre-process to filtering − Probabilistic filtering − Preserves texture detail Our Method − Preserves illumination edges − Suitable for progressive rendering 6

Algorithm Overview Radiance Radiance Radiance Unbiased image layers Compute vertex Trace paths contributions Incident Incident radiance Irradiance Perform radiance layers clustering Low variance buffers Iterate High intensity noise removal Probabilistic Filtered image filtering 7

Layer based approach Radiance = + + ��������� � � 1 � � 2 � � 3 Irradiance = + + 8

Layer based approach Radiance = + + ��������� � � 1 � � 2 � � 3 Irradiance = + + − Irradiance contributions: − Separate vertex contributions: − Ignores visible texture and BRDF − Minimise interference − Already available! − Simplifies feature detection 9

Radiance separation � � � � � � � � � ��� � · � � � � � � � � � � � , � � � �� � , � � , � � ��� � � � … � � � � � , � � , � � � �� � , � � , � � � � � ��� � · � � � � � � � � � � � � , � � � � · � � � � � �� � , � � , � � ����� − Utilise texture buffer: − Incident radiance radiance

11 v 3 � Clustering v = 2 v = 1

Clustering � v = 1 v = 2 v 3 − 1D Luminance domain: − Improves performance/storage − H.V.S. sensitive to luminance − Preserves visual discontinuities 12

Clustering � v = 1 v = 2 v 3 − Clusters at each layer: Cluster{ float[3] radiance float[3] irradiance − Aggregate samples min max float min float max − Represents (ir)radiance for sub-integral int numSamples : 31 bits int outlier : 1 bit − Contiguous, non-overlapping in 1D } 13

14 v 3 � Clustering v = 2 v = 1

Clustering − Heuristic based on: − Absolute sample difference − Cluster gradient − Add new contributions − Extend existing clusters

Clustering − Based on: − Absolute sample difference − Cluster gradient − Add new contributions − Extend existing clusters

Clustering − Based on: − Absolute sample difference − Cluster gradient − Add new contributions − Extend existing clusters − Merge clusters − Reduce memory costs − Dynamic approximation of integral − Always preserves largest discontinuities

Algorithm Overview Radiance Radiance Radiance Unbiased image layers Compute layer Trace paths contributions Incident Incident radiance Irradiance Perform radiance layers clustering Low variance buffers Iterate High intensity noise removal Probabilistic Filtered image filtering 18

Noise Removal − Density estimation approach − Image space neighbourhood − DeCoro et al. ’10 − Pixel-wise estimates − Pajot et al. ’11 − Key differences: − Filtering Pre-process: − Ignore texture variation − Prefer sparse data to no data − Layer based rejection − Accept if isolated and low energy Lower rejection rates Energy preservation More data 19

Illumination-aware filtering − Extends cross-bilateral filter: − Eisemann & Durand ’04 − Petschnigg et al. ’04 − Low variance range buffer − Poisson probability distribution: � � � � � � � � �� �! − Derive weights from cluster statistics − Probabilistic detection of illumination edges

Illumination-aware filtering Radiance Low var. buffers Find similar Calculate Scale cluster Final pixel clusters filter weights contributions radiance Incident Filtered image radiances 21

Find similar clusters − Extend min and max of cluster from pixel p − Find overlapping clusters in neighbouring contributing pixels, c � � � 5 � � � �� � � � � � − Compute poisson probabilities � � � ! using sample counts � � � 1 � � � 6 − Repeat for each cluster in pixel p

Calculate filter weights Radiance Radiance Geometric Low var. Compute Gaussian weights buffers geom. weights weights Radiance Radiance Poisson Filter weights kernel weights − Product of Gaussian, geometric and poisson weights: � � � � · � � · � � Poisson Gaussian Geometric 23

Scale cluster contributions − Radiance is a product of: − Texture , BRDF coefficient , incident radiance , � � � � � − Preserve detail in from pixel � � − Filter and using neighbouring pixels � � � − Weight with kernel � � � � � ��� · � � ��� · � � ��� � � � · � � � · � �� · � � � � � � � � � � ��� � � �

Results − Multiple coloured lights − Hard and soft gradients − Chromatic and luminance noise Reference Cross-Bilateral Path Tracing Our Method 25 16 samples per pixel

Results Path Tracing (22 samples) Reference Our method (16 samples) 26 Equal time comparison after 45 seconds

Results Cross-Bilateral Cross-Bilateral Reference Our method (Smooth noise) (Preserve texture) 27 Equal time comparison after 45 seconds

Results Path Tracing Our method Cross-Bilateral Reference − Self-shadowing − Complex geometry 28 Equal time comparison after 45 seconds

Conclusion − Fast image-based incident radiance filtering − Preserve illumination edges and gradients − Preserve texture detail − Remove high intensity noise − Future Work − Perceptual clustering heuristic − Layer specific parameters − Automatic kernel radius 29

Thank you for listening! Questions? (and Answers) 30

Thank you for listening! Questions? (and Answers) 31

32

Future Improvements − Aliasing: − Yang et al. ’11, Anti-Aliasing Recovery, ACM Trans. Graph. − Convergence: − Variance based adaptive filter widths − Noise removal parameters: − Difficult to establish for low sample counts − What is the pixels normal distribution? 33