Shape from X: perspective, texture, shading Thurs. Feb. 15, 2018 1 - PowerPoint PPT Presentation

COMP 546 Lecture 11 Shape from X: perspective, texture, shading Thurs. Feb. 15, 2018 1

Level of Analysis in Perception high - behavior: what is the task ? what problem is being solved? - brain areas and pathways - neural coding - neural mechanisms low 2

Level of Analysis in Perception high The last lecture and next few are more at this level. - behavior: what is the task ? what problem is being solved? - brain areas and pathways - neural coding - neural mechanisms low 3

3D Surface and Space Perception • Depth (of a point) -- talked a lot about this • Layout (of a scene) • Shape (of an object) 4

Perspective & vanishing points (where parallel lines meet) 5

Parallel lines that define a vanishing point can be in more than one 3D plane. 6

Man-made environments typically have three vanishing points 7

8

They will intersect at a finite point. 9

10

Q: What is the task ? What problem is being solved? A: 11

Q: What is the task ? What problem is being solved? A: To group local edges & lines that are are consistent with a vanishing point. These edges & lines can be perceived as parallel in the 3D scene . 12

COMP 546 Lecture 11 Shape from X: perspective, texture, shading Thurs. Feb. 15, 2018 13

Classically…. “texture” refers to the material (includes pigment & roughness) “shading” refers to lighting (includes shadows) 14

Texture From a book by Brodatz 15

Depth gradients from regular texture More than just vanishing points! 16

Depth gradients from random texture Fronto-parallel plane Slanted ground plane 17

Q: What is the task ? What problem is being solved? A: Judge the depth gradient from the image. 18

What is the depth gradient? 19

What is the depth gradient? 20

Depth map of a scene plane 𝑎 = 𝑎 0 + 𝐵𝑌 + 𝐶𝑍 ( 𝑌, 𝑍, 𝑎) is position of point on plane. 𝑍 (X, Y, Z ) 𝑌 𝑎 21

Depth gradient on a scene plane 𝑎 = 𝑎 0 + 𝐵𝑌 + 𝐶𝑍 𝜖𝑎 𝜖𝑌 , 𝜖𝑎 𝛼 𝑎 = = (𝐵, 𝐶) 𝜖𝑍 𝑍 (X, Y, Z ) 𝑌 𝑎 22

‘Slant’ (𝜏) Slant 𝜏 is defined to be the angle between 𝑎 = 𝑎 0 plane and the oblique plane. 𝑎 = 𝑎 0 𝑎 = 𝑎 0 + 𝐵𝑌 + 𝐶𝑍 𝐵 2 + 𝐶 2 𝜏 𝛼 𝑎 = 𝑎 = 𝑎 0 + 𝐵𝑌 + 𝐶𝑍 ≡ tan( 𝜏 ) 23

‘Tilt’ (𝜐) 𝜐 𝜐 is the direction of depth gradient. Thumbtack on plane 𝛼 𝑎 = 𝛼 𝑎 (cos 𝜐 , sin 𝜐 ) = tan ( 𝜏 ) (cos 𝜐 , sin 𝜐 ) 24

Examples slant ~ 45 deg slant ~ 0 deg slant ~ 45 deg tilt ~ 90 deg tilt undefined tilt ~ 30 deg 25

Slant and Tilt -225 -180 -135 -90 -45 0 45 90 5 15 25 35 45 55 65 75 85 26 (Koenderink, van Doorn, Kappers 1992)

Texture cues for slant & tilt • size gradient • density gradient • foreshortening gradient One can derive mathematical relationships for these quantities. Details omitted. 27

e.g. size and density gradient only (no foreshortening) 28

Slant and Tilt on a Curved Surface (foreshortening, density?, size?) Texture elements are compressed in the tilt direction, by an amount that depends on slant. 29

Slant and Tilt on a Curved Surface Cylinder Sphere all tilts and all slants tilt = 180 tilt = 0 30 all slants

Surface Curvature concavity cylinder hyperbolic cylinder convex (valley) (saddle) (ridge) (hill) [Koenderink and van Doorn 1992] Classical formal mathematical definitions of curvature are based on 2 nd derivatives. Details omitted. 31

Curvature of a face Local shape Curvedness 32

“Shape” from texture Q: What is the task ? What problem is being solved? A: Judge the slant, tilt, curvature across the surface. It is unknown how these scene shape properties are represented in the brain. 33

Shape from shading Drawings of Leonardo da Vinci 34

Shape from shading (random shape) 35

Surface normal 3D vector perpendicular to local tangent plane at general surface point. (𝑌 𝑞 , 𝑍 𝑞 , 𝑎 𝑞 ) Recall plane passing through Z axis: 𝑎 = 𝑎 0 + 𝐵𝑌 + 𝐶𝑍 𝜖𝑌 , 𝜖𝑎 𝜖𝑎 𝛼 𝑎 = = (𝐵, 𝐶) 𝜖𝑍 36

Surface normal 𝑌 𝑞 +∆𝑌, 𝑍 𝑞 +∆𝑍, 𝑎 𝑞 +∆𝑎 3D vector perpendicular to local tangent plane (𝑌 𝑞 , 𝑍 𝑞 , 𝑎 𝑞 ) = 𝑎 𝑞 + 𝜖𝑎 𝜖𝑌 ∆𝑌 + 𝜖𝑎 𝑎 𝑌 𝑞 +∆𝑌, 𝑍 𝑞 +∆𝑍 𝜖𝑍 ∆𝑍 + 𝐼. 𝑃. 𝑈. 37

Surface normal 𝑌 𝑞 +∆𝑌, 𝑍 𝑞 +∆𝑍, 𝑎 𝑞 +∆𝑎 3D vector perpendicular to local tangent plane (𝑌 𝑞 , 𝑍 𝑞 , 𝑎 𝑞 ) = 𝑎 𝑞 + 𝜖𝑎 𝜖𝑌 ∆𝑌 + 𝜖𝑎 𝑎 𝑌 𝑞 +∆𝑌, 𝑍 𝑞 +∆𝑍 𝜖𝑍 ∆𝑍 + 𝐼. 𝑃. 𝑈. ∆𝑎 ≈ 𝜖𝑎 𝜖𝑌 ∆𝑌 + 𝜖𝑎 𝜖𝑍 ∆𝑍 This vector is perpendicular to local tangent plane. 𝜖𝑎 𝜖𝑎 𝜖𝑌 , 𝜖𝑍 , −1 ∙ ∆𝑌, ∆𝑍, ∆𝑎 ≈ 0 38

Unit Surface Normal 1 𝜖𝑎 𝜖𝑌 , 𝜖𝑎 𝑂 ≡ 𝜖𝑍 , −1 2 2 𝜖𝑎 + 𝜖𝑎 + 1 𝜖𝑌 𝜖𝑍 𝑂 39

Shading on a sunny day  𝑂 L  N(x) 𝑀 Lambert’s (cosine) Law: I(x) = N(x) L

Shape from shading Q: What is the task ? What problem is being solved? A: 41

Shape from shading Q: What is the task ? What problem is being solved? (Why is it difficult to solve?) A: Judge : • lighting direction (?), • surface slant, tilt (?) • curvature (?) • surface normal (?) 42