Global Illumination Shadow Layers Franois Desrichard , David - PowerPoint PPT Presentation

Global Illumination Shadow Layers François Desrichard , David Vanderhaeghe, Mathias Paulin IRIT, Université de Toulouse, CNRS, INPT, UPS, UT1, UT2J, France July 12, 2019

Cinematic Lighting Design Scene Render Composite Animated geometry Diffuse, specular, Textures, materials normal, depth... Light rig Arbitrary Output Variables (AOVs) allow editing without re-rendering We prov rovid ide sh shadow laye yers rs for for com ompositing with little le ove overh rhead 2

Shadows Strong visual cue, subject to artistic expression Tolerant perception of shadow appearance (Hecher et al., 2014, Sattler et al., 2005) Horizon-Based Ambient Occlusion (HBAO) in Destiny 2 Nightmare by Alla Chernova 3

Previous Work Dragging with mouse On-surface deformation (Pellacini et al., 2002) (Ritschel et al., 2010) Rotation, pattern inlay (Obert et al., 2010) Shape simplification (DeCoro et al., 2007) 4

Comparison with Available Renderers Main layer = original image Arnold / pbrt / Cycles, no shadow Ours, no shadow Light overestimation ● Empirical matte ● No indirect shadows ● Arnold, shadow matte Ours, shadow layer 5

Comparison with Current Renderers Main layer Ours, shadow layer Arnold, shadow matte Compositing with the shadow layer 6

Contributions Definition of shadow layer under global illumination ● Characterization of the shadow layer in the path space ● Path tracer rendering all layers in a single pass ● 7

Defining Shadow Interaction between three components Light Caster Catcher The user defines caster and catcher by their surface How to account for indirect shadows? 8

Screen Space Definition Occlusion can be direct or indirect Main layer I Invisible render T Black body render B 9

Screen Space Definition As the difference between the two altered renders: S = T - B = - Shadow layer S Invisible render T Black body render B 10 10

Screen Space Definition Adding the shadow layer removes shadows + = Shadow layer with matte Main layer No shadow 11 11

Discussion on this Definition Agnostic to the rendering algorithm ● Takes into account indirect shadows ● Has a physical meaning: lost radiance ● Self-shadowing cannot be recovered ● No control on light or catcher ● Two additional renders per object ● 12 12

Translation to the Path Space Path integral formulation for the main layer (Veach, 1997) For the shadow layer of caster C = { all geometric light paths in the scene } = { light paths encountering caster C } = measurement contribution function considering C invisible 13 13

Integration in a Path Tracer Input: N objects as casters C i ● Output: the main layer and N shadow layers in a single pass ● The algorithm measures with , but also each with ● i i } Measuring with ⇔ scattering on all casters ○ p = 1 / 2 Measuring with ⇔ skipping caster C i ○ i 14 14

Integration in a Path Tracer 1. Intersect green 2. Skip green now 3. Contribute to S green 4. Cannot skip blue S blue ? S green 15 15

Integration in a Path Tracer 1. Contribute to S green 2. Intersect blue 3. Scatter on blue now 4. Contribute to I ? 16 16

Improved Artistic Control Per-light separation Main layer Separate light sources; shadow ratio I / ( I + S ) 17 17

Improved Artistic Control Direct and indirect separation Main layer Direct and indirect shadow ratio 18 18

Improved Artistic Control Custom catchers and self-shadowing toggle Main layer With and without self-shadows 19 19

Editing Examples Balancing shadow strength 20 20

Editing Examples Main layer + Locally color graded image Hibiscus / Pandanus shadow layer 21 21

Editing Examples Shape transformation using the shadow ratio I / ( I + S ) Main layer I ( I + S ) ⨯ the transformed shadow ratio 22 22

Performance Overhead Additional intersection tests bring the most overhead ● Managing samples and filtering incurs a per-caster cost ● The sampling budget is now shared among all layers ● For 1 to 5 casters, we measure a 1.1 to 1.3 overhead factor 23 23

Limitations When a shadow is created by multiple casters, it cannot be assigned to only one of them. A possible solution: consider them as one. Main layer Boxes as separate casters Both boxes as one caster With N interacting casters, combinatorial explosion (2 N shadow layers) 24 24

Limitations Using the notion of surface to define an object is limiting: how to handle participating media? Main layer No absorption No scattering 25 25

Conclusion Definition of shadow layer under global illumination ● Editable at the post-processing stage ○ Characterization of the shadow layer in the path space ● Amenable to Monte-Carlo integration algorithms ○ Path tracer rendering all layers in a single pass ● With an overhead in time and render convergence ○ 26 26

Thank you for your attention https://www.irit.fr/STORM/site/shadow-layers https://github.com/frcsdes/shadow-layers 27 27

References (DeCoro et al., 2007) DECORO C., COLE F., FINKELSTEIN A., RUSINKIEWICZ S.: Stylized shadows. In Proc. 5th International Symposium on Non- photorealistic Animation and Rendering (New York, NY, USA, 2007), NPAR ’07, ACM, pp. 77– 83. doi:10.1145/1274871.1274884. (Hecher et al., 2014) HECHER M., BERNHARD M., MATTAUSCH O., SCHERZER D., WIMMER M.: A comparative perceptual study of soft-shadow algorithms. ACM Trans. Appl. Percept. 11, 2 (July 2014). doi:10.1145/2620029. (Obert et al., 2010) OBERT J., PELLACINI F., PATTANAIK S.: Visibility editing for all-frequency shadow design. Computer Graphics Forum 29, 4 (2010), 1441 – 1449. doi:10.1111/j.1467-8659.2010.01741.x. (Pellacini et al., 2002) PELLACINI F., TOLE P., GREENBERG D. P.: A user interface for interactive cinematic shadow design. ACM TOG 21, 3 (July 2002), 563 – 566. doi:10.1145/566654.566617. (Ritschel et al., 2010) RITSCHEL T., THORMÄHLEN T., DACHSBACHER C., KAUTZ J., SEIDEL H.-P.: Interactive on-surface signal deformation. ACM TOG 29, 4 (July 2010), 36:1 – 36:8. doi:10.1145/1778765.1778773. (Sattler et al., 2005) SATTLER M., SARLETTE R., MÜCKEN T., KLEIN R.: Exploitation of human shadow perception for fast shadow rendering. In Proc. 2nd symposium on Applied perception in graphics and visualization (2005), APGV05, Association for Computing Machinery, p. 131 – 134. doi:10.1145/1080402.1080426. (Veach, 1997) VEACH E.: Robust Monte Carlo Methods for Light Transport Simulation. PhD thesis, Stanford University, Stanford, CA, USA, 1997. AAI9837162. 28 28

Scattered Layer Usually exported using a light path expression: .* <[RT].’tallbox’> .* = - Scattered layer Main layer I Black body render B 29 29

Scattered Layer 30 30

Participating Media Shadow layer test Shadow removal test 31 31

Integration in a Path Tracer 1. Propagation The first time a caster C is encountered, the path now belongs to Measure with ⇒ skip the surface and contribute to S ● C is assigned to the path and always skipped ○ No other caster may be skipped ○ Measure with ⇒ scatter normally and contribute to I ● C will never be skipped ○ Other casters can still be assigned ○ Both outcomes have probability p = 1 / 2 and introduce a factor 1 / p 32 32

Integration in a Path Tracer 2. Direct lighting If the path was assigned a caster the shadow ray can skip it, and contributes radiance to the corresponding shadow layer Otherwise: An unoccluded shadow ray contributes to I ● An occluded shadow ray can skip a caster that was never ● encountered and contribute to its shadow layer Overall, less zero radiance paths 33 33

Skip Probability Global illumination Direct illumination Direct shadows Our choice Global shadows p = 0 p = 1 / 2 p = 1 Future work: locally adapt p to the probability of an encounter? 34 34

Performance Table Equal sampling Equal time Equal variance Scene N SPP Time SSIM ZRP Time SPP μ -Var Var Time μ -SPP 2048 15’ 27” 0.903 26% 15’ 2048 0.003 0.01 18’ 24” 2321 0 1 2048 17’ 27” 0.901 17% 15’ 1824 0.004 0.01 20’ 08” 2318 Teaser 2048 18’ 07” 0.896 12% 15’ 1728 0.004 0.01 21’ 46” 2360 2 2048 18’ 28” 0.896 10% 15’ 1696 0.004 0.01 22’ 14” 2371 3 0 4096 27’ 53” 0.927 91% 30’ 9856 0.153 0.1 34’ 58” 8122 Dragon 4096 33’ 33” 0.889 90% 30’ 8256 0.153 0.1 41’ 44” 8139 1 0 1024 34’ 30” 0.992 87% 30’ 960 0.007 0.01 39’ 36” 1277 1024 42’ 35” 0.992 81% 30’ 768 0.007 0.01 51’ 11” 1289 1 Island 1024 46’ 05” 0.992 80% 30’ 704 0.008 0.01 54’ 16” 1310 3 5 1024 47’ 20” 0.992 79% 30’ 704 0.008 0.01 55’ 56” 1317 256 03’ 19” 0.901 15% 10’ 832 0.271 0.5 4’ 01” 291 0 Flowers 256 03’ 28” 0.899 12% 10’ 784 0.271 0.5 4’ 15” 291 1 35 35

Performance Table Teaser Dragon Island Flowers 36 36

Implementation Guidelines Bidirectional Path Tracing 1. Propagation Altered as path tracing for camera and light sub-paths 1. Integration The connection step can ignore occlusion Only form full paths with coherent propagation history: for instance, a sub-path that scattered on C cannot be connected to one that skipped C 37 37