Genesis: Real-Time Raytracing in Virtual Production FRANCESCO GIORDANA Realtime Architect MARCO GIORDANO Senior Developer
Technicolor has the industry’s most prominent portfolio of VFX brands, including MPC, MILL FILM and MR. X, Our VFX studios add VISUAL WONDER and CREATIVE EXPERTISE to film and episodic. Each studio brings their own unique approach to the market while leveraging Technicolor’s R&D teams to harness the new technological demands of compelling
VISUAL EFFECTS GLOBAL FOOTPRINT LONDON MONTREAL VANCOUVER TORONTO LOS ANGELES MUMBAI ADELAIDE BANGALORE MPC FILM MR X MILL FILM TECHNICOLOR
GENESIS
VR SCOUTING
MOCAP
VIRTUAL CAMERA
LAYERING
Phases of Virtual Production 1) Scout the environment, find shots, dress your set 2) Capture performance, add animations 3) Shoot virtual cameras 4) Iterate and layer
LED WALLS
LED Walls
Rendering Challenges 1) Advanced lighting for DP on set 2) Good approximation of final render 3) Final pixels in real time
AREA LIGHTS
VXGI
VOLUMETRIC LIGHTING
DEPTH OF FIELD
Not enough • Need more parity with lighting in post • Need raytracing: • Live link with a path tracer • Raytracing in engine
Live-Link with Prism
S9197 – Prism & RTX Victor Yudin, Lead Software Developer, Mill Film Wednesday, Mar 20, 9:00 AM - 09:50 AM SJCC Room 230B (Concourse Level)
Live link with RenderMan XPU • Same framework as Prism (Millefiori) • Prism is a delegate for Hydra, can replace with RenderMan delegate • USD enables loading the scene both in engine and in Millefiori • USD Shade for transferring materals, but still needs lots of work (MaterialX, MDL)
Raytracing in Unity HDRP
DXR INTEGRATION DETAILS
RAYTRACING IN GENESIS The goal: Leverage DXR to improve the quality of our - renders Different FPS targets based on the use case - The challenges: New to DX12 and DXR - Small team - Unity low level integration - New tech, with possible bugs to deal with -
AMBIENT OCCLUSION
AMBIENT OCCLUSION First effect implemented Plugin: boils down to a C interface dll Faced the first issues with Unity integration: • Data marshalling • Synchronization • Delayed reaction of editor events • Issue with resources lifetime • Integration of gameworks denoisers
AO: TIMINGS 1 spp 4spp 16 spp raytracing 2.2 4.3 17.4 denoise 2.1 2.1 2.1 AO 16 spp 4spp 1 spp 0 5 10 15 20 25 raytracing denoise
HDRP INTEGRATION • Unity use an AO renderer manager • Harder to integrate without drilling down in the call stack • Opted for simpler blit over Unity texture • Render moves on as usual • No tricks for increasing contrast
SOFT SHADOWS
AMBIENT OCCLUSION
SOFT SHADOWS • Heavy use of area lights • Previously using in house solution with raymarched depth map.
HDRP INTEGRATION • Did not find an easy way to get screen-space shadows • Modified HDRP shader binding extra shadow map • Multiply final lighting value by shadow • Crude approximation • Needed to extract texture from denoiser to C# texture to use it in Unity
Soft Shadows Experimenting with several ways to combine multiple are alights
AMBIENT OCCLUSION
Timings 1 spp 4 spp 16 spp raytracing 3.29 10.41 36.11 denoise 4.08 4.01 4.06 Shadows 16 spp 4 spp 1 spp 0 5 10 15 20 25 30 35 40 45 raytracing denoise
REFLECTIONS
Reflections Most tricky effect to deal with: • No way to evaluate a Unity shader on hit • Marshalling textures and textures lifetime • Lighting
Reflections • One bounce only • No texture marshalling • Investigating bindless textures • Only tracking directional light for simple Phong model • Extra shadow ray
Timings 1 SPP 4 SPP 16 SPP 2.3 3.01 17.9 Reflections 16 SPP 4 SPP 1 SPP 0 2 4 6 8 10 12 14 16 18 20
RECAP • DXR allowed us to step up quality of renders quite a bit • Steep learning curve for DX12 and DXR • Tricky Unity integration • Resource life time • Editor events • Render loop sync
WHAT IS NEXT • Investigate new Unity DXR integration • Focus more on effects rather than plumbing • Performance, performance, performance
Special thanks to the twitter community for RTX and game dev help: Matt Pettineo - @MyNameIsMJP Alex Tardif - @longbool Peter Morley - @Biblo_Shwag Sebastian Altonen - @SebAltonen Sebastien Lagarde - @SebLagarde Kostas Anagnostou - @KostasAAA Jesper Flicks - @catlikecoding Special thanks to Nvidia and the outstanding Microsoft Pix Team
THANK YOU!
THE ADVENTURE IS ABOUT TO BEGIN… WE’RE HIRING! APPLY AT www.moving-picture.com www.mpc-rnd.com OR EMAIL recruitment@moving-picture.com
Recommend
More recommend
