Image Based Lighting dr. Francesco Banterle - PowerPoint PPT Presentation

Image Based Lighting dr. Francesco Banterle francesco.banterle@isti.cnr.it

Image Based Lighting: why? • Image Based Lighting (IBL): • To (re)light synthetic objects with real-world lighting

Image Based Lighting: why? • IBL is very important: • advertisement: cars, forniture, etc. • visual effects: CGI, live motion, etc. • augmented reality • cultural heritage

IBL: Capturing Lighting • The input of IBL is real-world lighting • HDR imaging is the key • A HDR photograph captures a limited portion of light coming from the point of capture

IBL: Capturing Lighting

IBL: Capturing Lighting • Solution: • To capture HDR panoramic image 360x180 • These images are typically called either environment map or lightprobe

IBL: Capturing Lighting

IBL: Capturing Lighting • How capturing spherical (360x180) images? • Single shot panorama cameras: • SpheronVR: 50Mpix and 24 f-stops • iSTAR 360: 50Mpix and 27 f-stops • Roundshot: 160Mpix • These cameras may be expensive…

IBL: Capturing Lighting • to capture a mirror sphere (e.g. xmas ball) • to capture a panoramic image from multiple directions and exposure times. This requires post-processing; e.g. image stitching: • PTGui: • http://www.ptgui.com • Hugin (open source): • http://hugin.sourceforge.net/download/

IBL: Capturing Lighting

IBL: Capturing Lighting

IBL: Capturing Lighting 0.289

IBL: Capturing Lighting 0.289

IBL: Capturing Lighting 0.289

IBL: Capturing Lighting 0.289 0.537

IBL: Capturing Lighting y z θ φ x x

IBL: Longitude Latitude Mapping φ θ 2 3 ✓ ◆ sin θ cos φ j 1 − 1 � � θ = π 1 − 2 height 2 cos θ D = 4 5 π 2 i sin θ sin φ φ = width

IBL: Longitude Latitude Mapping • Advantages: • easy mapping to understand/implement • Disadvantages: • not equal-area —>pixels cover different areas on the sphere • squeezed at the poles —> to take this into account

IBL: angular mapping   cos φ sin θ sin φ sin θ D =   − cos θ φ = arctan(1 − 2 y, 2 x − 1) (2 x − 1) 2 + (2 y − 1) 2 p θ = π

IBL: angular mapping • Advantages: • avoiding undersampling at edges • Disadvantages: • not equal-area —>pixels cover different areas on the sphere • a bit more complicated

IBL: cube mapping 2 3 2 x − 1 1 2 y − 1 D = 4 5 1 + (2 x − 1) 2 + (2 y − 1) 2 p 1  1 �  1 � 3 , 2 2 , 3 x ∈ ∧ y ∈ . 3 4 y x

IBL: cube mapping 2 3 2 x − 1 1 2 y − 1 D = 4 5 1 + (2 x − 1) 2 + (2 y − 1) 2 p 1  1 �  1 � 3 , 2 2 , 3 x ∈ ∧ y ∈ . 3 4 y x

IBL: cube mapping • Advantages: • hardware support on the GPU • Disadvantages: • not equal-area (pixels are bigger at edges) —>pixels cover different areas on the sphere

… and now?

Rendering ~ ! o ~ ! i ~ n x

Rendering Z L o ( x , ~ ! o ) = L e ( x , ~ ! o ) + Ω + L i ( x , ~ ! i ) f r ( x , ~ ! o ) | ~ n · ~ ! i | d ~ ! i , ~ ! i

Rendering

Rendering Z L o ( x , ~ ! o ) = L e ( x , ~ ! o ) + Ω + L i ( ~ ! i ) f r ( x , ~ ! o ) | ~ n · ~ ! i | d ~ ! i , ~ ! i

Rendering Z L o ( x , ~ ! o ) = L e ( x , ~ ! o ) + Ω + L i ( ~ ! i ) f r ( x , ~ ! o ) | ~ n · ~ ! i | d ~ ! i , ~ ! i

Rendering • How to solve this integral? • Creating light sources from the environment map • Sampling the environment map

Light sources generation • Direction light sources are extracted from the environment map. • Properties: direction, and HDR color • The integral is converted into: N X L j ! j ! j L o ( x , ~ ! o ) = L e ( x , ~ ! o ) + i f r ( ~ ! o ) | ~ n · ~ i | i , ~ j =1 • Number of light sources is a parameter: more lights more time. Few lights —> bias (integral not converged)

Light sources generation: uniform sampling Subdivide the panorama in regular regions

Light source generation: uniform sampling Extracted light sources

Light sources generation: median-cut sampling Subdivide the panorama in regions of equal luminance

Light sources generation: median-cut sampling Subdivide the panorama in regions of equal luminance

Light sources generation: median-cut sampling Subdivide the panorama in regions of equal luminance

Light sources generation: median-cut sampling Subdivide the panorama in regions of equal luminance

Light sources generation: median-cut sampling Subdivide the panorama in regions of equal luminance

Light sources generation: median-cut sampling Subdivide the panorama in regions of equal luminance

Light sources generation: median-cut sampling Subdivide the panorama in regions of equal luminance

Light sources generation: median-cut sampling Subdivide the panorama in regions of equal luminance

Light source generation: median-cut sampling Extracted light sources

Light source generation: median-cut sampling Extracted light sources

Light source generation: median-cut sampling

Light source generation: median-cut sampling

Sampling the Environment Map • Solving: Z L o ( x , ~ ! o ) = L e ( x , ~ ! o ) + Ω + L i ( ~ ! i ) f r ( x , ~ ! o ) | ~ n · ~ ! i | d ~ ! i , ~ ! i • with monte-carlo methods; generating samples according to a probability distribution: N ! x j ! x j ! x j ! o ) + 1 L i ( ~ i ) f r ( ~ ! o ) | ~ n · ~ i | i , ~ X L o ( x , ~ ! o ) = L e ( x , ~ p ( x j ) N j =1 • Few samples —> noise

Sampling the Environment Map

Sampling the Environment Map

how to insert virtual objects?

Differential Rendering

Differential Rendering

Differential Rendering

Differential Rendering

Differential Rendering Images are courtesy of Karsch

Differential Rendering / Synthetic Objects + Support Geometry Only Support Geometry

Differential Rendering Shadows to insert into the photograph (table)

Differential Rendering x

Differential Rendering

Differential Rendering ( ) + x

Differential Rendering

there is more…

nobody expects… the Spanish Inquisition

Spatial IBL • A single environment can capture only distant light sources • Nearby light sources are not modeled as local light but as distant ones • To increase realism there is the need to model them properly

Spatial IBL

Spatial IBL

Spatial IBL

Spatial IBL

Spatial IBL

Questions?