RENDERING FASTER AND BETTER WITH VRWORKS John Spitzer, April 4, - PowerPoint PPT Presentation

April 4-7, 2016 | Silicon Valley RENDERING FASTER AND BETTER WITH VRWORKS John Spitzer, April 4, 2016

VR Ready Headset Hardware Developers Partners HELPING THE ECOSYSTEM BUILD AMAZING VR App Engine Developers Developers 2

LEADING ENGINES ADOPT NVIDIA VRWORKS Multi-Res Shading Multi-Res Shading VR SLI VR SLI 3

LEADING HMD MAKERS ADOPT VRWORKS Direct Mode Direct Mode Context Priority VR SLI 4

VR PLATFORMS FOR CONSUMER, PRO, AND MOBILE GeForce Quadro Tegra VR gaming and VR workstations Mobile VR headsets entertainment PCs Location-based VR Embedded video capture World’s only VR notebooks and streaming devices CAVEs Consumer 360 video capture 360 video capture/streaming 970 / 980 Ti / Titan X M5000M6000 5

Gaming Sports CONSUMER APPLICATIONS OF VR ARE HUGE Movies Concerts 6 Travel Retail

M&E Medical PROFESSIONAL VR MARKET MAY BE EVEN LARGER AEC Energy 7 Manufacturing Collaboration

VR IS RAMPING QUICKLY 600+ 40,000 $3.8B Companies Employees Capital Raised VR/AR = $80B revenue by 2025 8 Source: Goldman Sachs

HOW IS VR RENDERING DIFFERENT? 9

HOW IS VR RENDERING DIFFERENT? High framerate, low latency High FPS, low latency Stereo Rendering Lens Distortion 10

NVIDIA VRWORKS SDK App Headset Increase Scale performance Minimize head Plug and play Reduce latency by performance via an with multiple GPUs tracking latency compatibility rendering directly innovative new way with asynchronous from GPU to HMD to the front buffer to render for VR time warp Pro only WARP AND BLEND SYNCHRONIZATION GPU GPUDIRECT AFFINITY FOR VIDEO API for geometry and Provides tear free VR Fine grain control Reduces latency for intensity adjustments environments by to pin OGL contexts video transfer to for seamless VR synchronizing multi GPU’s to specific GPU’s and from the GPU 11

VR SLI Two eyes...two GPUs! 12

“ NORMAL ” SLI GPUs render alternate frames N N+1 CPU N GPU 0 N+1 GPU 1 N N+1 Display Latency 13

VR SLI Each GPU renders one eye — lower latency N N+1 CPU N L N+1 L GPU 0 N R N+1 R GPU 1 N N+1 Display Latency 14

VR SLI GPU affinity masking: full control Left eye rendering Shadow maps, GPU physics, etc. Right eye rendering 15

VR SLI Broadcasting reduces CPU overhead L Render scene once R 16

VR SLI Per-GPU constant buffers, viewports, scissors L L R Multi-GPU Engine API R 17

VR SLI Cross-GPU data transfer via PCI Express 18

MULTI-RESOLUTION SHADING 19

VR HEADSET OPTICS Distortion and counter-distortion 20

VR HEADSET OPTICS Distortion and counter-distortion User’s view Image Displayed Optics 21

DISTORTED RENDERING Render normally, then resample Distorted image Rendered image 22

DISTORTED RENDERING Over-rendering the outskirts Distorted image Rendered image 23

MULTI-RESOLUTION SHADING Subdivide the image, and shrink the outskirts 24

MULTI-RESOLUTION SHADING Fast viewport broadcast on NVIDIA Maxwell GPUs Viewport 1 Viewport 2 Geometry Pipeline ... Viewport N 25

MULTI-RESOLUTION SHADING Public, open-source SDK https://developer.nvidia.com/vrworks NVAPI for Maxwell viewport broadcast in DX11/12 Helper code for configuring viewports, shader math, etc DX11 sample app Guide to integrating in an existing engine OpenGL extensions are also available 26

UE4 INTEGRATION https://github.com/NvPhysX/UnrealEngine/tree/MultiRes-4.10 Postprocessing passes supported: Bloom & lens flare Tonemapping Temporal AA Reflection environment Height fog Decals Distortion/refraction 27

MULTI-RESOLUTION SHADING Performance Infiltrator demo seeing 30-40% increase Everest game seeing up to 40% increase Up to 50% when entirely pixel-bound! 28 NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE.

RENDERING PASSES Geometry passes: add FastGS; render with 9 viewports & scissors Base pass, decals, deferred lights, depth prepass, distortion, shadow projection, translucency, velocity Image-space passes: remap linear/multi-res UVs as needed Decals, deferred lights, postprocessing, shadow projection, tiled deferred lights Text/HUD would be done after multi-res 29

IMAGE-SPACE PASSES Two common cases Reconstruct 3D position from 2D+depth Map 2D position to linear before reconstructing Large filter kernels Map sample point to linear, apply offset, map back to multi-res 30

UV REMAPPING 31

DEMO TIME! Unreal Engine 4.11 32

NVIDIA VRWORKS SDK App Headset Increase Scale performance Minimize head Plug and play Reduce latency by performance via an with multiple GPUs tracking latency compatibility rendering directly innovative new way with asynchronous from GPU to HMD to the front buffer to render for VR time warp Pro only WARP AND BLEND SYNCHRONIZATION GPU GPUDIRECT AFFINITY FOR VIDEO API for geometry and Provides tear free VR Fine grain control Reduces latency for intensity adjustments environments by to pin OGL contexts video transfer to for seamless VR synchronizing multi GPU’s to specific GPU’s and from the GPU 33

Context priority for asynchronous time warp Reduces latency from head rotation Frame Frame Warped (Left) (Right) Frame Head Tracking (t) Head Tracking (t+1) 34

Direct Mode Plug-and-play Compatibility Via Direct Interface with the VR Headset Operating System 35

Front Buffer Rendering Reduces Latency By Rendering Directly to the Headset Scan Out Warp 36

NVIDIA VRWORKS SDK App Headset Increase Scale performance Minimize head Plug and play Reduce latency by performance via an with multiple GPUs tracking latency compatibility rendering directly innovative new way with asynchronous from GPU to HMD to the front buffer to render for VR time warp Pro only WARP AND BLEND SYNCHRONIZATION GPU GPUDIRECT AFFINITY FOR VIDEO API for geometry and Provides tear free VR Fine grain control Reduces latency for intensity adjustments environments by to pin OGL contexts video transfer to for seamless VR synchronizing multi GPUs to specific GPU’s and from the GPU 37

Warp and blend APIs for rendering custom immersive environments API for Geometry, Intensity and Black Offset on each display Adjustments applied to the full desktop image Zero latency Application independent Image courtesy of Joachim Tesch — Max Planck Institute for Biological Cybernetics 38

Synchronization Prevent tearing and image misalignment in VR environments Synchronize multiple displays Co-ordinate Buffer Swaps Frame Lock: Align the scan out of Swap Groups and Swap Barriers: multiple displays, GPU’s, and systems HW based swap synchronization within a node or between clusters Stereo Lock: Maintain Stereo alignment between multiple systems Synchronize to an internal or external timing source 39

Enables real-time, low latency video transfers in VR environments 360 degree video capture and stitching Capture, render & output in less than 3 frames Synchronized Transfers Minimal CPU Overhead GPUDirect ™ Sub Frame Transfer Size for video 40

NVIDIA VRWORKS: PAVING THE WAY FOR VR App Headset Increase Scale performance Minimize head Plug and play Reduce latency by performance via an with multiple GPUs tracking latency compatibility rendering directly innovative new way with asynchronous from GPU to HMD to the front buffer to render for VR time warp Pro only WARP AND BLEND SYNCHRONIZATION GPU GPUDIRECT AFFINITY FOR VIDEO API for geometry and Provides tear free VR Fine grain control Reduces latency for intensity adjustments environments by to pin OGL contexts video transfer to for seamless VR synchronizing multi GPU’s to specific GPU’s and from the GPU 41

QUESTIONS? 42

April 4-7, 2016 | Silicon Valley THANK YOU JOIN THE NVIDIA DEVELOPER PROGRAM AT developer.nvidia.com/join