Effects Techniques Used in Uncharted 3: Drakes Deception Marshall - PDF document

Effects Techniques Used in Uncharted 3: Drake’s Deception Marshall Robin Graphics/Effects Programmer, Naughty Dog <mrobin@naughtydog.com> @mrobin604 Monday, March 12, 12 Good afternoon, and welcome to my talk on the effects techniques used in Uncharted 3: Drake’s Deception. My name is Marshall Robin, I’m a visual effects programmer at Naughty Dog. I’ve been at Naughty Dog since 2005, and I’ve been working on the Uncharted series since Drake’s Fortune.

Overview • Goals for effects system • Tools • Runtime • Example - Sand Footprints Monday, March 12, 12 Design goals Description and demonstration of tools used by VFX artists Get into runtime architecture - data structures and spu job chain Usage example - sand prints. Surface projection shader.

Particles • Scheme based macro language • Ubershader • Processing split between PPU and SPU SPU Update Cull Sort PPU Calc Process Build Render Stall! Stall! Monday, March 12, 12 E fg ects system used in Uncharted 3 began development after UDF UDF system was di ffj cult to use, felt that to produce quality & quantity of fx, we needed to rework. FX artists write FX in scheme based macro language, rebuild, upload. Hard to visualise. Shader was 6000+ line ubershader, brittle and di ffj cult to use, so most e fg ect shaders were very simple. On runtime side, processing was split between PPU and SPU, stalling on the PPU for jobs to finish. OK for initial PS3 title, but we needed to be more e ffj cient for sequel!

New System • Drastically improve iteration time • Faster iteration == more effects & polish • More data driven • Better dynamics • More flexible shaders with modern features • 100% asynchronous SPU code Monday, March 12, 12 These issues informed design for new system Iteration - artists should see changes when they build Data - more artist control. Curves, ramps, custom data Dynamics - add fields to change velocity Shaders - more modular, easier to use & modify, new features - distortion, soft particles Move rest of code to SPUs, remove sync points. 2.5 to 4x bonus from naive port.

Tools • Particler (Particle authoring) • Noodler (Shader creation) Monday, March 12, 12

Particler Monday, March 12, 12 Particle authoring tool Runs on maya, written in Python using Maya UI commands U1 was di ffj cult to get a new FX artist up to speed, we wanted tool artists could use right away Basing on maya was obvious choice Particles created using Maya’s particle nodes (emitters, shapes, fields). Ramps and curves attached to most parameters. Particle expressions. Show demo. Doug creates a single emitter smoke e fg ect with an animated turbulence field.

Monday, March 12, 12 Create e fg ect by placing a set of emitters Each emitter independently configured Emitter, particle shape, field windows (Describe workflow?) A particle artist creates an e fg ect by compositing multiple simple particle elements in Maya. Each element has its own configuration, the shaders, fields, and emitters can all be separate. The final e fg ect is placed and spawned as a single unit known as a particle group. (other stu fg to talk about) The artist can attach fields to the emitters. Fields are similar to the fields found in Maya particles - they modify the current velocity of the particle in some way. For example, a drag field will decelerate the particle in proportion to it’s current velocity. A gravity field will accelerate the particle in a specified direction by a fixed acceleration. 2 minute demo video here with basic operation of Particler? Items to include in the demo: Create emitters and shapes, and attach them Create fields and attach to emitters Edit animation of emitter Edit a color ramp Edit script Show and hide di fg erent elements Build and show each part in game Info from particler slides to work into the narration. Discuss some but not all! Emitters: Point, Volume; Continuous, burst, and distance emission; Speed & Lifespan w/random variance; Inherit velocity from parent Fields: Types: Gravity, Drag, Air, Radial, Vortex, VolumeAxis, Turbulence, Kill; Volumes for attenuation; Most parameters can be animated Curves & Ramps: Curves stored as cubic splines; Curves are used with emitters and fields; Ramps can be used with particle expressions Expressions: Applied to particle state; Artist can define extra state vars; Expressions compiled to byte code (more on this later)

Building the Effect • Information extracted from nodes for export • Particle definitions • Fields • Curve & Ramp Tables • Expressions • Compile expressions into VM byte code • Write all data out as DC format Monday, March 12, 12 When e fg ect built, Maya scene is traversed and game data is gathered from emitters and all attached shape and field nodes. Curves and ramps attached to attributes are collected in tables. Expressions are parsed and compiled to VM byte code Data written in DC file

DC • Data Compiler • Used for most runtime data • Scheme based (For more info, check out Dan Liebgold’s GDC 2008 presentation) Monday, March 12, 12 DC - human readable format used for gameplay data. File gets converted into binary by the Data Compiler tool, hence DC. Built on top of Scheme. Dan Liebgold gave a presentation at GDC in 2008 talking all about it.Available in slide and audio in the GDC Vault.

Particler to Game • Particler spawns interactive DC session with plugin • DC persists to speed up future builds for quick iteration .DC File Network Display Particler DC Engine Monday, March 12, 12 Once the DC file is created, Particler then calls a custom Maya plugin that launches an interactive DC session and connects to it via a pipe. Particler then sends the DC session commands to compile and upload the file to the game. DC has a network connection directly to the runtime, and can send it commands to reload binary data files, replacing the copy that it already has. The runtime then resets the spawned particles and uses this new copy of the file to respawn the edited e fg ect. The new e fg ect is then displayed for the artist, who can quickly review his changes. The DC session is kept alive to process further commands from Particler, so iteration time is very quick and requires no more from the artist than tweaking whatever items they want to change and hitting the build button. So that’s Particler.

Noodler • Rewritten shaders still not flexible enough • Sony ATG demos editor for us - let’s try it Monday, March 12, 12 Artists could not experiment Node based shaders = slow and hard to debug? Write shaders for U2 but overwhelmed with artist requests Mike D asks for node based editor so he can write his own shaders No, shaders too slow and hard to maintain. Right before U3, Sony ATG shows material system with NBE Decide to give it a shot since it was ez to integrate

Noodler Monday, March 12, 12 Noodler - node based editor. It’s written in C++. All vertex inputs set up Color and alpha outputs to interface node Can use external lighting as defined by game Provided nodes: math, texture, attribute, ??? Can define nodes by writing cg functions (cool!) Short demo putting together a shader and bringing it in game

Monday, March 12, 12 Demo video here for noodler! Ideas: Build a simple blend shader Drop into surfer, connect up textures Preview

Caustic Shader Monday, March 12, 12 After editor integrated into tool pipeling, gave to Eben Cook to try and get feedback Within a day, he had written this shader <show noodler graph of caustic> Shader fakes water caustics reflected onto a surface Previously used flipbook, required a lot of texture memory This shader uses no textures for caustics, and only requires 19 cycles Here’s the final result: <show video of caustic shader, compared with the flipbook>

Monday, March 12, 12

Noodler Internals • Vertex frontend - define input attributes • Interface node - outputs fed into lighting code • Each node is a Cg function • Automatically split into VS & FS • Automatic space conversion Monday, March 12, 12 Vertex frontend - defines functions that convert vertex registers to attributes used inside the tool Interface node - output sink for the graph, used by lighting backend to render final color Each node is turned into a Cg function Automatically checks type agreement Automatically splits VS & FS, but you can override Handles attribute passing between VS and FS by swizzling VS outputs automatically Handles converting space changes for vectors and points

How did it go? • Noodler used for all FX shaders in U3! • Programmer help needed for some debugging • Decreased over time as artists learned more • Made artists manage their cycle budgets Monday, March 12, 12 Super fast iteration Artists could create unique shaders that improved the dynamic look of their e fg ects Programmer help with numerical issues, render state, some debugging Cycle counts were fairly reasonable

Spawning Effects • Spawners in Charter (Level Editor) • Script • Animation effect files (EFF) • Water Spawners • (directly in code too) Monday, March 12, 12 Once loaded, how do you create? Environmental e fg ects - spawned when camera enters an associated region Animations - Specify keyframe and joint Script - using spawner location Code - not used much but available (projectiles)

Charter Spawning Monday, March 12, 12

Script Spawning Monday, March 12, 12

Animation Spawning Monday, March 12, 12

Water Spawners Monday, March 12, 12