R Real-Time Rendering l Ti R d i (Echtzeitgraphik) - PowerPoint PPT Presentation

R Real-Time Rendering l Ti R d i (Echtzeitgraphik) (Echtzeitgraphik) Michael Wimmer wimmer@cg tuwien ac at wimmer@cg.tuwien.ac.at

Walking down the graphics pipeline i li Application Application Geometry Geometry Rasterizer Rasterizer

What for? Understanding the rendering pipeline is the key to real-time rendering! key to real-time rendering! Insights into how things work Understanding algorithms Understanding algorithms Insights into how fast things work Performance Vienna University of Technology 3

Simple Graphics Pipeline Often found in text books Will t k Will take a more detailed look into OpenGL d t il d l k i t O GL Application Application Geometry Geometry Rasterizer Rasterizer Display Display Display Display Vienna University of Technology 4

Graphics Pipeline (pre DX10, OpenGL 2 ) Nowadays, everything Application CPU CPU part of the pipeline is part of the pipeline is C Driver hardware accelerated try try Command Command Geomet Geomet Geometry G Fragment: “pixel”, but with additional info Rasterization er er asterize asterize ( (alpha, depth, stencil, …) ) Texture Ra Ra Fragment Di Display l Vienna University of Technology 5

Fixed Function Pipeline – Dataflow View video memory on-chip cache memory vertex pre-TnL geometry shading cache ( (T&L) ) system commands memory post-TnL cache CPU CPU triangle setup rasterization rasterization texture fragment textures cache shading shading and frame buffer raster operations ti Vienna University of Technology 6

DirectX10 /OpenGL 3.2 Evolution Vertex Application Input CPU CPU Buffer Assembler Index C Buffer Buffer Driver Vertex Texture Shader try try Command Command Geomet Geomet Geometry Texture Shader Geometry G St Stream Buffer Out Rasterization er er Setup/ Memory asterize asterize Rasterization Texture Pixel Texture Ra Ra Shader Shader Fragment Depth Output Merger Merger Di Display l C l Color Vienna University of Technology 7

OpenGL 3.0 OpenGL 2.x is not as capable as DirectX 10 But : New features are vendor specific But : New features are vendor specific extensions (geometry shaders, streams…) GLSL a little more restrictive than HLSL (SM 3.0) OpenGL 3.0 did not clean up this mess! p p  OpenGL 2.1 + extensions  Geometry shaders are only an extension  Geometry shaders are only an extension  New: depreciation mechanism OpenGL 4 x OpenGL 4.x  New extensions  OpenGL ES compatibility!  OpenGL ES compatibility! Vienna University of Technology 8

DirectX 11/OpenGL 4.0 Evolution fixed f fixed programmable programmable memory memory Constant Constant Constant Constant Constant Constant Constant Constant Control Control Input Input Vertex Vertex Geometry Geometry Setup Setup Pixel Pixel Output Output Point Point Tessellator Tessellator Assembler Assembler Shader Shader Shader Shader Rasterizer Rasterizer Shader Shader Merger Merger Shader Shader Stream Stream Sampler Sampler Sampler Sampler Sampler Sampler Sampler Sampler out out Vertex Vertex Index Index S S tream tream Depth Depth R R ender ender Texture Texture Texture Texture Texture Texture Texture Texture Buffer Buffer Buffer Buffer Buffer Buffer S S tencil tencil Target Target Memory Memory Memory Memory Vienna University of Technology 9

DirectX 11 Tesselation At unexpected position! At unexpected position! Compute Shaders Multithreading To reduce state change overhead To reduce state change overhead Dynamic shader linking y g HDR texture compression M Many other features... th f t Vienna University of Technology 10

11 DirectX 11 Pipeline Vienna University of Technology

Application Generate database (Scene description) Usually only once Usually only once Load from disk Build acceleration structures (hierarchy, …) B ild l ti t t (hi h ) Simulation (Animation, AI, Physics) ( y ) Input event handlers Modify data structures Modify data structures Database traversal Primitive generation Shaders (vertex,geometry,fragment) Shaders (vertex,geometry,fragment) Vienna University of Technology 12

Driver Maintain graphics API state C Command interpretation/translation d i t t ti /t l ti Host commands  GPU commands Handle data transfer Memory management M t Emulation of missing hardware features g U Usually huge overhead! ll h h d! Significantly reduced in DX10 g y Vienna University of Technology 13

Geometry Stage Command Command Vertex Processing Vertex Processing Vertex Processing Vertex Processing Tesselation Tesselation Tesselation Tesselation Primitive Assembly Primitive Assembly Geometry Shading Geometry Shading Geometry Shading Geometry Shading Clipping Clipping P P Perspective Division Perspective Division ti ti Di i i Di i i Culling Culling Culling Culling Vienna University of Technology 14

Command Command buffering (!) Unpack and perform format conversion format conversion Command Command (“Input Assembler”) interpretation glLoadIdentity( ); glLoadIdentity( ); glMultMatrix( T ); glMultMatrix( T ); Color glBegin( GL TRIANGLE STRIP ); g glBegin( GL TRIANGLE STRIP ); g g g ( ( _ _ _ _ ) ) T T Transformation matrix glColor3f ( 0.0, 0.5, 0.0 ); glColor3f ( 0.0, 0.5, 0.0 ); glVertex3f( 0.0, 0.0, 0.0 ); glVertex3f( 0.0, 0.0, 0.0 ); glColor3f ( 0.5, 0.0, 0.0 ); glColor3f ( 0.5, 0.0, 0.0 ); glColor3f ( 0.5, 0.0, 0.0 ); glColor3f ( 0.5, 0.0, 0.0 ); glVertex3f( 1.0, 0.0, 0.0 ); glVertex3f( 1.0, 0.0, 0.0 ); glColor3f ( 0.0, 0.5, 0.0 ); glColor3f ( 0.0, 0.5, 0.0 ); glVertex3f( 0.0, 1.0, 0.0 ); glVertex3f( 0 0 glVertex3f( 0.0, 1.0, 0.0 ); glVertex3f( 0 0 1 0 1 0 0 0 ); 0 0 ); glColor3f ( 0.5, 0.0, 0.0 ); glColor3f ( 0.5, 0.0, 0.0 ); glVertex3f( 1.0, 1.0, 0.0 ); glVertex3f( 1.0, 1.0, 0.0 ); glEnd( ); glEnd( ); lE d( ) lE d( ) Vienna University of Technology 15

Vertex Processing Transformation Vertex Processing Vertex Processing normalized eye object clip window device d i v e Modelview Modelview Projection Projection Perspective Perspective Viewport Viewport r r t Matrix Matrix Division Transform e x Modelview Projection Modelview Modelview    Vienna University of Technology 16

Vertex Processing Fixed function pipeline: User has to provide matrices the rest happens User has to provide matrices, the rest happens automatically Programmable pipeline: Programmable pipeline: User has to provide matrices/other data to shader h d Shader Code transforms vertex explicitly We can do whatever we want with the vertex! Usually a gl_ModelViewProjectionMatri x is provided In GLSL-Shader : gl_Position = ftransform(); Vienna University of Technology 17

Vertex Processing Lighting T Texture coordinate generation and/or t di t ti d/ transformation Vertex shading for special effects T Screen-space lit triangles Object-space triangles Vienna University of Technology 18

Tesselation If just triangles, nothing needs to be done, otherwise: otherwise: Evaluation of polynomials for curved surfaces  Create vertices (tesselation) DirectX11 specifies this in hardware! DirectX11 specifies this in hardware! 3 new shader stages!!! Still not trivial (special algorithms required) Vienna University of Technology 19

20 DirectX11 Tesselation Vienna University of Technology

Optimally tesslated! 21 Tesselation Example Vienna University of Technology

Geometry Shader Calculations on a primitive (triangle) A Access to neighbor triangles t i hb t i l Limited output (1024 32-bit values) p ( )  No general tesselation! Applications : A li i Render to cubemap p Shadow volume generation T i Triangle extension for ray tracing l t i f t i Extrusion operations (fur rendering) p ( g) Vienna University of Technology 22

Rest of Geometry Stage Primitive assembly G Geometry shader t h d Clipping (in homogeneous coordinates) pp g ( g ) Perspective division, viewport transform C lli Culling Vienna University of Technology 23

Rasterization Stage Triangle Setup T i T i Triangle Setup l l S t S t R R i i i i Rasterization Rasterization Fragment Fragment Texture Texture Processing Processing Processing Processing Processing Processing Processing Processing Raster Operations Raster Operations Raster Operations Raster Operations Vienna University of Technology 24

Rasterization Setup (per-triangle) S Sampling (triangle = {fragments}) li (t i l {f t }) Interpolation (interpolate colors and p ( p coordinates) Screen-space triangles Fragments Vienna University of Technology 25

Rasterization Sampling inclusion determination I In tile order improves cache coherency til d i h h Tile sizes vendor/generation g specific Old graphics cards: 16x64 Old graphics cards: 16x64 New: 4x4  Smaller tile size favors conditionals in shaders  All tile fragments calculated in parallel on modern hardware on modern hardware Vienna University of Technology 26