Illumination and Reflectance Tues. Feb. 20, 2018 1 Illumination - PowerPoint PPT Presentation

COMP 546 Lecture 12 Illumination and Reflectance Tues. Feb. 20, 2018 1

Illumination and Reflectance • Shading • Brightness versus Lightness • Color constancy

Shading on a sunny day  𝑂 L  N(x) 𝑀 Lambert’s (cosine) Law: 𝐽 𝑌 = 𝑂(𝑌) ∙ 𝑀

Unit Surface Normal 1 𝜖𝑎 𝜖𝑌 , 𝜖𝑎 𝑂 ≡ 𝜖𝑍 , −1 2 2 𝜖𝑎 + 𝜖𝑎 + 1 𝜖𝑌 𝜖𝑍 𝑂 4

Shading on a sunny day 5

Cast and Attached Shadows attached : 𝑂(𝑌) ∙ 𝑀 < 0 𝑀 cast : 𝑂(𝑌) ∙ 𝑀 > 0 but light is occluded

Shading models • sunny day (last lecture) • sunny day + low relief • cloudy day

Examples of low relief surfaces (un)crumpled paper

Low relief surface 𝜖𝑎 𝜖𝑎 ≈ 0 ≈ 0 Suppose and are both small. 𝜖𝑌 𝜖𝑍 Thus, 1 𝜖𝑌 , 𝜖𝑎 𝜖𝑎 𝑂 ≡ 𝜖𝑍 , −1 2 2 𝜖𝑎 + 𝜖𝑎 + 1 𝜖𝑌 𝜖𝑍 ≈ 0 ≈ 0 9

Linear shading model for low relief 𝜖𝑎 𝜖𝑎 𝐽 𝑌, 𝑍 ≈ , , −1 ∙ 𝑀 𝑌 , 𝑀 𝑍 , 𝑀 𝑎 𝜖𝑌 𝜖𝑍 • shadows can still occur • one equation per point but two unknowns 10

Example: curtains 𝑌 𝑎 𝑌, 𝑍 = 𝑎 0 + 𝑏 sin( 𝑙 𝑌 𝑌 ) 𝑎 𝑀 𝑂

Example: curtains 𝑌 𝑎 𝑌, 𝑍 = 𝑎 0 + 𝑏 sin( 𝑙 𝑌 𝑌 ) 𝑎 𝜖𝑎 𝜖𝑎 = 𝑏 𝑙 𝑌 cos 𝑙 𝑌 𝑌 = 0 , 𝜖𝑌 𝜖𝑍

Example: curtains 𝜖𝑎 𝜖𝑎 𝐽 𝑌, 𝑍 ≈ , , −1 ∙ 𝑀 𝑌 , 𝑀 𝑍 , 𝑀 𝑎 𝜖𝑌 𝜖𝑍 𝑎 𝑌, 𝑍 = 𝑎 0 + 𝑏 sin( 𝑙 𝑌 𝑌 ) 𝜖𝑎 𝜖𝑎 = 𝑏 𝑙 𝑌 cos 𝑙 𝑌 𝑌 = 0 , 𝜖𝑌 𝜖𝑍 𝐽 𝑌, 𝑍 = 𝑏 𝑙 𝑌 cos 𝑙 𝑌 𝑌 𝑀 𝑌 − 𝑀 𝑎

Example: curtains 𝑌 𝑎 𝑌, 𝑍 = 𝑎 0 + 𝑏 sin( 𝑙 𝑌 𝑌 ) 𝐽 𝑌, 𝑍 = − 𝑀 𝑎 + 𝑏 𝑙 𝑌 𝑀 𝑌 cos 𝑙 𝑌 𝑌 𝑎 𝑀 = (𝑀 𝑌 , 𝑀 𝑍 , 𝑀 𝑎 ) 𝑂 Q: where do the intensity maxima and minima occur?

Shading on a cloudy day (my Ph.D. thesis) 15

Shading on a Cloudy Day  (x) Shadowing effects cannot be ignored.

Shading on a Cloudy Day Shading is determined by shadowing and surface normal. 17

Shading on a Sunny Day Shading determined by surface normal only.

Shape from shading Q: What is the task ? What problem is being solved? A: Estimate surface slant, tilt, curvature. How to account for (or estimate) the lighting ? 19

Illumination and Reflectance • Shading [Shading and shadowing models assume that intensity variations on a surface are entirely due to illumination. But surfaces have reflectance variations too. ] • Brightness versus Lightness • Color constancy

Which paper is lighter ? Paper on the left is in shadow. It has lower physical intensity and it appears darker. But do both papers seem to be of same (white) material? 21

Which paper is lighter ? Image is processed so that the right paper is given same image intensities as left paper. Now, right paper appears to be made of different material. Why? 22

𝐽 𝑦, 𝑧 = 𝑗𝑚𝑚𝑣𝑛𝑗𝑜𝑏𝑢𝑗𝑝𝑜 𝑦, 𝑧 ∗ 𝑠𝑓𝑔𝑚𝑓𝑑𝑢𝑏𝑜𝑑𝑓 (𝑦, 𝑧) “Real” example Abstract version

Physical quantities luminance 𝐽 𝑦, 𝑧 = 𝑗𝑚𝑚𝑣𝑛𝑗𝑜𝑏𝑢𝑗𝑝𝑜 𝑦, 𝑧 ∗ 𝑠𝑓𝑔𝑚𝑓𝑑𝑢𝑏𝑜𝑑𝑓 (𝑦, 𝑧) shading & shadows material

𝐽 𝑦, 𝑧 = 𝑗𝑚𝑚𝑣𝑛𝑗𝑜𝑏𝑢𝑗𝑝𝑜 𝑦, 𝑧 ∗ 𝑠𝑓𝑔𝑚𝑓𝑑𝑢𝑏𝑜𝑑𝑓 (𝑦, 𝑧) low reflectance, high illumination low reflectance, low illumination high reflectance, low illumination

Perceptual quantities “brightness” 𝐽 𝑦, 𝑧 = 𝑗𝑚𝑚𝑣𝑛𝑗𝑜𝑏𝑢𝑗𝑝𝑜 𝑦, 𝑧 ∗ 𝑠𝑓𝑔𝑚𝑓𝑑𝑢𝑏𝑜𝑑𝑓 (𝑦, 𝑧) (no standard term for “lightness” perceived illumination)

Adelson’s corrugated plaid illusion. All four indicated “squares” have same intensity. What is the key difference between the two configurations ?

“Lightness” perception Q: What is the task ? What problem is being solved? A:

“Lightness” perception Q: What is the task ? What problem is being solved? A: Estimate the surface reflectance , by discounting the illumination. 𝐽 𝑦, 𝑧 = 𝑗𝑚𝑚𝑣𝑛𝑗𝑜𝑏𝑢𝑗𝑝𝑜 𝑦, 𝑧 ∗ 𝑠𝑓𝑔𝑚𝑓𝑑𝑢𝑏𝑜𝑑𝑓 (𝑦, 𝑧) ?

“Lightness” perception: solution sketch Q: What is the task ? What problem is being solved? A: Estimate the surface reflectance , by discounting illumination effects. Compare points that have same illumination. 𝐽 𝑦 1 , 𝑧 1 = 𝑗𝑚𝑚𝑣𝑛𝑗𝑜𝑏𝑢𝑗𝑝𝑜 ∗ 𝑠𝑓𝑔𝑚𝑓𝑑𝑢𝑏𝑜𝑑𝑓 (𝑦 1 , 𝑧 1 ) = 𝐽 𝑦 2 , 𝑧 2 = 𝑗𝑚𝑚𝑣𝑛𝑗𝑜𝑏𝑢𝑗𝑝𝑜 ∗ 𝑠𝑓𝑔𝑚𝑓𝑑𝑢𝑏𝑜𝑑𝑓 (𝑦 2 , 𝑧 2 )

Illumination and Reflectance • Shading • Brightness versus Lightness • Color constancy

Recall lecture 3 - color Absorption Illumination by photoreceptors (fraction) Surface Reflectance (fraction) There are three different spectra here. 32

LMS cone responses Cone response 𝐽 𝑦, 𝑧, 𝜇 𝐷 𝑀𝑁𝑇 (𝜇) 𝑒𝜇 𝐽 𝑦, 𝑧, 𝜇 = 𝑗𝑚𝑚𝑣𝑛𝑗𝑜𝑏𝑢𝑗𝑝𝑜 𝑦, 𝑧, 𝜇 ∗ 𝑠𝑓𝑔𝑚𝑓𝑑𝑢𝑏𝑜𝑑𝑓 𝑦, 𝑧, 𝜇

Surface Color Perception Q: What is the task? What is the problem to be solved? A: illumination Surface Reflectance 34

Surface Color Perception Q: What is the task? What is the problem to be solved? A: Estimate the surface reflectance, by discounting the illumination . illumination Surface Reflectance 𝐽 𝑦, 𝑧, 𝜇 = 𝑗𝑚𝑚𝑣𝑛𝑗𝑜𝑏𝑢𝑗𝑝𝑜 𝑦, 𝑧, 𝜇 ∗ 𝑠𝑓𝑔𝑚𝑓𝑑𝑢𝑏𝑜𝑑𝑓 𝑦, 𝑧, 𝜇 35

For simplicity, let’s ignore the continuous wavelength 𝜇 and just consider 𝑆𝐻𝐶 ( LMS ). Cone response 𝐽 𝑦, 𝑧, 𝜇 𝐷 𝑀𝑁𝑇 (𝜇) 𝑒𝜇 𝜇 𝐽 𝑆𝐻𝐶 𝑦, 𝑧 = 𝑗𝑚𝑚𝑣𝑛𝑗𝑜𝑏𝑢𝑗𝑝𝑜 𝑆𝐻𝐶 𝑦, 𝑧 ∗ 𝑠𝑓𝑔𝑚𝑓𝑑𝑢𝑏𝑜𝑑𝑓 𝑆𝐻𝐶 (𝑦, 𝑧)

“Color Constancy” Task: estimate the surface reflectance, by discounting the illumination illumination Surface Reflectance 𝐽 𝑆𝐻𝐶 𝑦, 𝑧 = 𝑗𝑚𝑚𝑣𝑛𝑗𝑜𝑏𝑢𝑗𝑝𝑜 𝑆𝐻𝐶 𝑦, 𝑧 ∗ 𝑠𝑓𝑔𝑚𝑓𝑑𝑢𝑏𝑜𝑑𝑓 𝑆𝐻𝐶 (𝑦, 𝑧) 37

Why we need color constancy • object recognition • skin evaluation (health, emotion, …) • food quality • …

Example 1: spatially uniform illumination = * 𝐽 𝑆𝐻𝐶 𝑦, 𝑧 = 𝑗𝑚𝑚𝑣𝑛𝑗𝑜𝑏𝑢𝑗𝑝𝑜 𝑆𝐻𝐶 𝑦, 𝑧 ∗ 𝑠𝑓𝑔𝑚𝑓𝑑𝑢𝑏𝑜𝑑𝑓 𝑆𝐻𝐶 (𝑦, 𝑧) Given this, how to estimate this?

Solution 1: ‘max - RGB’ Adaptation = * 𝐽 𝑆𝐻𝐶 𝑦, 𝑧 = 𝑗𝑚𝑚𝑣𝑛𝑗𝑜𝑏𝑢𝑗𝑝𝑜 𝑆𝐻𝐶 𝑦, 𝑧 ∗ 𝑠𝑓𝑔𝑚𝑓𝑑𝑢𝑏𝑜𝑑𝑓 𝑆𝐻𝐶 (𝑦, 𝑧) Divide each 𝐽 𝑆𝐻𝐶 channel by the max value of 𝐽 𝑆𝐻𝐶 in each channel. When does this give the correct answer?

Example 2: non-uniform illumination sun + only blue sky blue sky = * 𝐽 𝑆𝐻𝐶 𝑦, 𝑧 = 𝑗𝑚𝑚𝑣𝑛𝑗𝑜𝑏𝑢𝑗𝑝𝑜 𝑆𝐻𝐶 𝑦, 𝑧 ∗ 𝑠𝑓𝑔𝑚𝑓𝑑𝑢𝑏𝑜𝑑𝑓 𝑆𝐻𝐶 𝑦, 𝑧 Solution: See Exercises.

Illumination and Reflectance • Shape from Shading • Brightness versus Lightness • Color constancy Different solutions require different underlying assumptions. This is separate issue from how we can code up a solution using neural circuits.