I ntroduction to Programming I ntroduction to Programming Mapping - PowerPoint PPT Presentation

I ntroduction to Programming I ntroduction to Programming Mapping Techniques On The GPU Mapping Techniques On The GPU Cliff Lindsay Cliff Lindsay Ph.D. Student, C.S. WPI http://users.wpi.edu/~ clindsay � [images courtesy of Nvidia and Addision-Wesley]

Motivation Motivation Why do we need and want mapping? Why do we need and want mapping? •Realism •Ease of Capture vs. Manual Creation •GPUs are Texture Optimized (Texture = Efficienct Storage) Metal Using Metal Using Solid Color Metal Solid Color Metal Mapping Techniques Mapping Techniques [Images from Pixar]

Quest for Visual Realism

Talk Overview Talk Overview • Review Basic Texturing Review Basic Texturing • • Environment Mapping Environment Mapping • Bump Mapping • Bump Mapping • • Displacement Mapping Displacement Mapping •

Texture Mapping Texture Mapping Main Idea: Use an image to apply color to the pixels Produce by geometry of an object. [Catmull 74] Model Model Texture Texture Render Render

I s it simple? I s it simple? � Idea is simple---map an image to a surface--- there are 3 or 4 coordinate systems involved 2D image 3D surface

Texture Mapping Texture Mapping parametric coordinates texture coordinates window coordinates world coordinates

Mapping Functions Mapping Functions � Basic problem is how to find the maps � Consider mapping from texture coordinates to a point a surface � Appear to need three functions x = x(s,t) (x,y,z) y = y(s,t) z = z(s,t) t � But we really want s to go the other way

Backward Mapping Backward Mapping � We really want to go backwards � Given a pixel, we want to know to which point on an object it corresponds � Given a point on an object, we want to know to which point in the texture it corresponds � Need a map of the form s = s(x,y,z) t = t(x,y,z) � Such functions are difficult to find in general

Texture and Texel Texture and Texel � Each Pixel in a Texture map = Texel � Each Texel has (u,v) 2D Texture Coordinate � Range of (u,v) is [0.0,1.0] (normalized)

Are there I ssues? Are there I ssues? 2 Problems: 2 Problems: • Which Texel should we use? Which Texel should we use? • • Where Do We Put Texel? Where Do We Put Texel? • Sample Sample Locations Locations Texture Map Texture Map 2 Solutions: 2 Solutions: Sampling & Filtering Sampling & Filtering •Map >1 Texel to 1 Coordinate Map >1 Texel to 1 Coordinate • •Nearest, Interpolation, & More Nearest, Interpolation, & More • Texels Texels Coordinate Generation Coordinate Generation a) UV (most common) b) Spherical c) Cylindrical d) Planar a) b) c) d)

(u,v) tuple tuple (u,v) � For any (u,v) in the range of (0-1, 0-1) multiplied by texture image width and height , we can find the corresponding value in the texture map

How to get F(u,v)? How to get F(u,v)? � We are given a discrete set of values: � F [i,j] for i= 0 ,…,N, j= 0 ,…,M � Nearest neighbor: � F � F (u,v) = F [ round(N* u), round(M* v) ] � Linear Interpolation: � i = floor(N* u), j = floor(M* v) � interpolate from F [i,j], F [i + 1 ,j], F [i,j + 1 ], F [i + 1 ,j] � Filtering in general !

I nterpolation I nterpolation Nearest neighbor Linear Interpolation

UV Coordinates For Triangles UV Coordinates For Triangles � Given a triangle defined by three points ( a, b, a, b, c ), how do we associate a texture color with a c point on the triangle? c ? b a

Computing the Point Computing the Point � Given the (x,y) point in the triangle, how do we transform that to a (u,v) point in the image? � Set up a non-orthogonal coordinate system with origin a and basis vectors b - a and c - a γ = 2 c a γ = 1 - c b a - b a 0 γ = 1 0 1 - = = = β β β

Barycentric coordinates coordinates Barycentric � Any point on the triangle can be defined by the barycentric coordinate p = a + β (b-a) + γ (c-a) γ = 2 c p γ = 1 a - c b a - b a 0 γ = 1 0 1 - = = = β β β

Barycentric coordinates coordinates Barycentric β,γ ) barycentric coordinate for the � Once we have computed the ( β,γ triangle, we can determine the corresponding ( u, v u, v ) point. � First, establish the ( u, v u, v ) system: (0, 1) (1, 1) (0, 0) (1, 0)

Computing the (u, v) coordinate Computing the (u, v) coordinate � u( β, γ ) = u a + β (u b – u a ) + γ (u c – u a ) � v( β, γ ) = v a + β (v b – v a ) + γ (v c – v a )

Example: Texture Example: Texture Mapping On GPU Mapping On GPU Assumptions : Assumptions •We Have Existing Geometry •Texture Coordinates Pre-generated •Texture map We Can Write 2 Shaders: We Can Write 2 Shaders: Vertex • Vertex – Set Geometry & Pass Through Coordinates • • Fragment Fragment – Sample Texture & Apply Pixel to Shading •

Example: Texture Example: Texture Mapping On GPU Mapping On GPU Vertex Shader Vertex Shader st r uct Ver t _O st r uct Ver t _O ut put { ut put { f l oat 4 posi t i on : PO f l oat 4 posi t i on : PO SI TI O SI TI O N; N; f l oat 3 col or : CO f l oat 3 col or : CO LO LO R; R; f l oat 2 f l oat 2 t exCoor d : TEXCO t exCoor d : TEXCO O O RD0; RD0; } ; } ; Ver t _O Ver t _O ut put ut put ver t _shader ( ver t _shader ( f l oat 2 posi t i on : PO f l oat 2 posi t i on : PO SI TI O SI TI O N, N, f l oat 3 col or : CO f l oat 3 col or : CO LO LO R, R, f l oat 2 t exCoor d : TEXCO f l oat 2 t exCoor d : TEXCO O O RD0) RD0) { Ver t _O Ver t _O ut put O ut put O UT; UT; O O UT. posi t i on = f l oat 4( posi t i on, 0, 1) ; UT. posi t i on = f l oat 4( posi t i on, 0, 1) ; O O UT. col or = col or ; UT. col or = col or ; O O UT. t exCoor d = t exCoor d; UT. t exCoor d = t exCoor d; r et ur n O r et ur n O UT; UT; }

Example: Texture Example: Texture Mapping On GPU Mapping On GPU Fragment Shader Fragment Shader st r uct f r ag_O st r uct f r ag_O ut put { ut put { f l oat 4 col or : CO f l oat 4 col or : CO LO LO R; R; } ; } ; f r ag_O f r ag_O ut put ut put f r ag_shader ( f r ag_shader ( f l oat 2 t exCoor d : TEXCO f l oat 2 t exCoor d : TEXCO O O RD0, RD0, uni f or m uni f or m sam sam pl er 2D decal : TEX0) pl er 2D decal : TEX0) { f r ag_O f r ag_O ut put O ut put O UT; UT; O O UT. col or = t ex2D( decal , UT. col or = t ex2D( decal , t exCoor d) ; t exCoor d) ; r et ur n O r et ur n O UT; UT; }

Applying Our Mapping knowledge Applying Our Mapping knowledge Further Realism Improvements: Further Realism Improvements: Environment Mapping • Environment Mapping • • Bump Mapping Bump Mapping • • Displacement Mapping Displacement Mapping • • Illumination Mapping & Others? Illumination Mapping & Others? •

Environment Mapping Environment Mapping Main idea: “Environment Maps are textures that describe, for all directions, the incoming or out going light at a point in space.” [Real Time Shading, pg. 49]” Reflections from Reflections from Environment Environment Three main types : Three main types � Cube Mapping � Sphere mapping � Paraboloid Mapping No Map applied Map Applied No Map applied Map Applied [Images courtesy of Microsoft, msdn.microsoft.com]

Environment Mapping Environment Mapping Cubic Mapping � Camera takes orthographic pictures in six axis � (-X, X, Y, -Y, Z, -Z) � = Calculating a reflection vector Map Look Up = X, Y, Z X, Y, Z I.E.: R = (3.14, .21, -8.7) Z is largest Z is largest & negative & negative Cube Texture Map Cube Texture Map [image courtesy of NVidia.com] * Index into the Negative Z region (dark blue)

Environment Mapping Environment Mapping Sphere Mapping � Generated from photographing a reflective sphere � Captures whole environment Sphere Texture Map Sphere Texture Map [Diagram and Sphere Map image of a Cafe in Palo Alto, CA, Heidrich]

Environment Mapping Environment Mapping Sphere Mapping r r r r r = − ⋅ ⋅ • � Obtain the reflection vector: R I 2 . 0 N ( N I ) Index into the Sphere map: R R 1 1 = + = + y x s , t m 2 m 2 ( ) ( ) = + + + 2 2 2 m 2 R R R 1 x y z [image courtesy of nVidia.com]

Environment Mapping Environment Mapping Paraboloid Mapping ( ) 1 1 + y ≤ = − + 2 2 2 2 x 1 f ( x , y ) x y , where 2 2 High Lights: � 2 textures, 1 per hemisphere � No artifacts at poles � Requires 2 passes or 2 texture fetches to render [ Shaded areas of Paraboloid Map , image adapted from [phd]]

Environment Mapping Environment Mapping Cons : � Sphere maps have a singularity of the parameterization of this method, we must fix viewing direction, view-dependent (meaning if you want to change the viewers direction you have to regenerate the Sphere map). � Paraboloid maps requires 2 passes Pros: � Better sampling of the texture environment for Paraboloid mapping, view-independent, � Cube maps can be fast if implemented in hardware (real-time generation), view independent,