PROFESSIONAL VR: AN UPDATE Robert Menzel, Ingo Esser GTC 2018, March 26 2018
NVIDIA VRWORKS Comprehensive SDK for VR Developers GRAPHICS TOUCH & PHYSICS HEADSET AUDIO PROFESSIONAL VIDEO 2
NVIDIA VRWORKS ADOPTION ENGINES HEADSETS SOME PRO APPLICATIONS 3
NVIDIA VRWORKS Comprehensive SDK for VR Developers GRAPHICS TOUCH & PHYSICS HEADSET AUDIO PROFESSIONAL VIDEO 4
CAD DATA IN VR 5
GRAPHICS PIPELINE VR Workloads 3x Application 124M Pix/s 1080 N vertices 60 Hz Preprocessing 1920 Geometric 3x Pipeline Rasterization ~3.6x Fragment Shader 457M Pix/s 1680 2N vertices 90 Hz Postprocessing 6 1512 1512
NVIDIA VRWORKS Comprehensive SDK for VR Developers GRAPHICS TOUCH & PHYSICS HEADSET AUDIO PROFESSIONAL VIDEO 7
SINGLE PASS STEREO Traditional Rendering Render eyes separately Doubles CPU and GPU load 8
SINGLE PASS STEREO Using SPS to improve rendering performance Single Pass Stereo uses Simultaneous Multi-Projection architecture Draw geometry only once Vertex/Geometry stage runs once Outputs two positions for left/right Only rasterization is performed per-view 9
SINGLE PASS STEREO OpenGL In OpenGL via GL_NV_stereo_view_rendering Create texture array for rendering left and right eye simultaneously No other changes needed, shaders perform SPS 10
SINGLE PASS STEREO OpenGL - Vertex Shader Calculate projection space position proj_pos = proj * view * model * inPosition; Output both positions via different builtin variables, only x component may differ gl_Position = proj_pos + vec4(offset, 0, 0, 0); gl_SecondaryPositionNV = proj_pos – vec4(offset, 0, 0, 0); Use declaration and value of gl_Layer to route output to layers 0 and 1 of tex array layout(secondary_view_offset=1) out highp int gl_Layer; gl_Layer = 0; 11
SINGLE PASS STEREO Vulkan In Vulkan via VK_NVX_multiview_per_view_attributes Create layered texture image and view for rendering left and right simultaneously Requires MultiView support Update: VK_KHX_multiview ratified to VK_KHR_multiview in Vulkan 1.1 12
SINGLE PASS STEREO Vulkan - Vertex Shader Calculate projection space position proj_pos = (proj * view * model * inPosition).xyz; Standard MultiView – specify once, may execute shader twice gl_Position = proj_pos + UBO.offsets[gl_ViewIndex]; With per-view attributes - also specify positions explicitly, execute shader only once gl_PositionPerViewNV[0] = proj_pos + UBO.offsets[0]; gl_PositionPerViewNV[1] = proj_pos + UBO.offsets[1]; 13
GRAPHICS PIPELINE Single Pass Stereo Performance Results SPS Application Single Pass Stereo: Benefits in geometry bound scenarios Preprocessing Heavy fragment shaders will reduce scaling 7.2 7.2 7.2 7.1 7.1 7.1 6.9 6.8 6.8 6.7 6.7 6.7 Geometric SPS Pipeline 4.9 4.5 4.5 3.7 3.7 3.7 Rasterization Fragment Shader Postprocessing Flat shading Flat shading Flat shading + Phong + Phong + Noise Traditional Traditional Traditional MultiView MultiView MultiView MultiView with per-view attributes MultiView with per-view attributes MultiView with per-view attributes 14 NVIDIA Quadro P6000, Scene with 17.6M faces, frame times in ms
NVIDIA VRWORKS Comprehensive SDK for VR Developers GRAPHICS TOUCH & PHYSICS HEADSET AUDIO PROFESSIONAL VIDEO 15
VR SLI Crash course Left view data L R Geometry Materials Right view data R 16
VR SLI Scaling 1 vs 2 GPUs App Left App Right App Both GPU L GPU L GPU R GPU R Copy Time: GPU L + GPU R Time: GPU + Copy 2 ∗ 𝐻𝑄𝑉 𝑇𝑑𝑏𝑚𝑗𝑜 = 𝐻𝑄𝑉 + 𝐷𝑝𝑞𝑧 17
VR SLI Scaling determined by workload and copy time 2 2 ∗ 𝐻𝑄𝑉 𝑇𝑑𝑏𝑚𝑗𝑜 = 1.8 x1.72 𝐻𝑄𝑉 + 𝐷𝑝𝑞𝑧 1.6 Typical render resolution for Vive 1.4 1.2 Scaling factor 1512 x 1680 (per eye) 1 Copy time over PCIe (@6GB/s) 0.8 0.6 1.5ms 0.4 Max scaling with 11ms frame time 0.2 0 2∗9.5𝑛𝑡 9.5𝑛𝑡+1.5𝑛𝑡 = 1.72 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Workload (ms) 9.5ms 18
VR SLI Higher resolutions limit scalability 2 2 ∗ 𝐻𝑄𝑉 𝑇𝑑𝑏𝑚𝑗𝑜 = 1.8 𝐻𝑄𝑉 + 𝐷𝑝𝑞𝑧 1.6 Vive Pro render resolution 1.4 1.2 Scaling Factor 2016 x 2240 1 Copy time over PCIe (@6GB/s) 0.8 0.6 2.8ms 0.4 Max scaling with 11ms frame time 0.2 2∗8.2𝑛𝑡 0 8.2𝑛𝑡+2.8𝑛𝑡 = 1.49 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Copy time (ms) 19
VR SLI Improve scaling using NVLink Copy times can hurt scaling with higer resolutions NVLink on dual Quadro GP100: 4x faster than PCIe 3.0 Copy time for Vive Pro (2016 x 2240): 0.7ms Max scaling with 11ms frame time 2∗10.3𝑛𝑡 10.3𝑛𝑡+0.7𝑛𝑡 = 1.87 NVLink is used automatically if present NVLINK speed measured with 2 bridges, copy via OpenGL multicast, single frame of HTC Vive, on HP z840 workstation 20
VR SLI Upcoming HMDs can improve scaling Some upcoming HMDs have one display cable per eye SLI system: Plug each cable into one GPU Eliminate inter-GPU copies by presenting on each GPU Needs support from the VR runtime: near future Working on 4-GPU configuration Two pairs of GPUs connected via NVLink One Multicast context spanning the configuration Split Frame Rendering (SFR), NVLink copies 21 HMD image courtesy of Starbreeze
OPENGL & VULKAN 22
OPENGL MULTICAST 2 Feedback on Multicast led to new functionality Command & data broadcast Dynamic Multicast toggle (WGL_NV_multigpu_context) BufferSubData to specific GPU GPU_ID built-in in GLSL shader CopyImageSubData & CopyBufferSubData Per-GPU viewports & scissors GPU-GPU Framebuffer Blit Texture & Buffer upload mask Global barrier & directed sync functions Asynchronous copies GPU Masks Per-GPU sample locations Per-GPU queries 23
OPENGL MULTICAST 2 Dynamic SLI mode New extension WGL_NV_multigpu_context: Request SLI mode per context No need to restart application Multicast Context Data Context Possible to share resources between contexts Left view Geometry Materials Right view 24
OPENGL MULTICAST 2 Dynamic SLI mode New extension WGL_NV_multigpu_context: Request SLI mode per context No need to restart application Multicast Context AFR Context Data Context Possible to share resources between contexts Left view Frame i On toggle: Geometry Clean up per-GPU resources Materials Keep scene data Right view Frame i+1 Alternate Frame Rendering (AFR) 25
MULTICAST 2 GPU ID built-in Multicast v1 required per-GPU uploads Larger code changes in some renderers Add shader built-in Left view Upload all views to all GPUs Geometry Geometry Use per-GPU data in shaders Materials Materials Renderer can remain unchanged Views Just modify shaders instead Right view 26
MULTICAST 2 Per-GPU Viewports & Scissors Add new function to set viewports and scissors per GPU Per-GPU Lens Matched Shading 27
MULTICAST 2 Per-GPU Viewports & Scissors Add new function to set viewports and scissors per GPU Per-GPU Lens Matched Shading Per-GPU Multi Resolution Shading 28
MULTICAST 2 Per-GPU Viewports & Scissors Add new function to set viewports and scissors per GPU Per-GPU Lens Matched Shading Per-GPU Multi Resolution Shading Easily set up Split Frame Rendering (SFR) 29
MULTICAST 2 Texture & Buffer Upload Mask Multicast provides per-GPU buffer uploads Asymmetrical functionality wrt texture upload functions Add new mask function to modify texture & buffer uploads glUploadGpuMaskNVX( GLbitfield mask ); Useful for simpler per-GPU texture streaming L Conserve PCIe bandwidth R 30
MULTICAST 2 Asynchronous Copies Multicast copies stall source GPU while copy takes place Easy to use because of implicit synchronization New copy functions do not stall, but also need more synchronization glAsyncCopyBufferSubDataNV (…) GPU L1 GPU L2 glAsyncCopyImageSubDataNV (…) Copy Copy while both GPUs can continue rendering Allows for more complex rendering algorithms GPU R1 GPU R2 31
MULTICAST 2 Asynchronous Copies – Use case Render shadow maps (SM) Start async copies of SMs to other GPU SM_0.. SM_i SM_i+1.. SM_N Render z-prepass per GPU & eye Z Right Z Left Wait for copy to finish Render output images SM_0.. SM_N 32
VR SLI Vulkan Update: VK_KHX_device_group ratified to VK_KHR_device_group with Vulkan 1.1 Make sure to use the right extension/Vulkan version combination! Usage is the same, so migration is painless Display Left view L L R Geometry Render Materials Right view R 33
VR SLI Recap Application VR SLI covers a wide variety of workloads Preprocessing Almost perfect load balancing between left/right eye and two GPUs Geometric Copy overhead and view independent VR SLI Pipeline workloads limit scaling NVLink can help improve scaling Rasterization Fragment Shader Dual-input HMDs can eliminate copy overhead Postprocessing 34
NVIDIA VRWORKS Comprehensive SDK for VR Developers GRAPHICS TOUCH & PHYSICS HEADSET AUDIO PROFESSIONAL VIDEO 35
HMD OPTICS Countering Lens Distortion Displayed Image Optics User’s View 36
HMD RENDERING Oversampling near the borders Rendered Image Displayed Image 37
LENS MATCHED SHADING Four Viewports Original Image LMS Image 38
Recommend
More recommend
