WetBrush: GPU-based 3D Painting Simulation at the Bristle Level - PowerPoint PPT Presentation

WetBrush: GPU-based 3D Painting Simulation at the Bristle Level Zhili Chen Byungmoon Kim Adobe Research GTC 2016

Oil painting with brush  Complex interactions • Bristle-Bristle • Bristle-Paint • Paint-Paint  Motivation: • Exhaustive simulation? 2 GTC 2016

Oil Painting Simulation Two-way Brush head Bristle-Paint Liquid movement Interaction User Input Brush Sim Paint Liquid Renderer 3 GTC 2016

Brush Model  Previous works on brush model Natural brushes in Photoshop DiVerdi et al., 2010 D-Brush in Photoshop Chu et al., 2002 Expresii 2D Surface wrapped Individual bristles 2D Stamps around skeleton • • • No variations from No details from individual Brush shape not realistic without brush dynamics bristles with large inter-bristle interactions deformation 4 GTC 2016

Brush Model  Our model Bristle vertices 𝐪 𝑗 Bristle samples 𝑚 0 𝐪 𝑗+1 Stretching 𝐪 𝑗−1 𝐪 𝑗 𝐪 𝑗+1 𝜄 Bending 50-100 bristles ~10 vertices per bristle 5 GTC 2016

Brush Model  Inter-bristle interactions • Collision • Friction 6 GTC 2016

Paint Liquid Model Height field Volumetric grid Particles Overhang Chu, et al. 2010 Microsoft FreshPaint ? • Track volume and mass better • Less blurring from grid-sampling • Model paint carried by brush Multi-layer color mixing 7 GTC 2016

Hybrid Fluid Representation  Adaptive Hybrid Fluid Representation based on • Distance to brush • Velocity Brush Fluid particles 8 GTC 2016

Hybrid Fluid Representation  Adaptive Hybrid Fluid Representation based on • Distance to brush • Velocity Particles Brush  Close to the brush Fluid  OR Fast Moving particles  Cover smaller region 9 GTC 2016

Hybrid Fluid Representation  Adaptive Hybrid Fluid Representation based on • Distance to brush • Velocity Particles Brush  Close to the brush Fluid  OR Fast Moving particles  Cover smaller region Density Grid  Further away from the brush  AND Slow moving  Cover larger region 10 GTC 2016

Fluid Conversions Grid & Particles Only Particles Visualized Visualized Particle Fluid • • Convert and delete Sample new stationary particles particles Grid Fluid  FLIP • Particles sample from velocity grid • Avoid expensive neighbor searching 11 GTC 2016

Hybrid Fluid • Less volume loss • Better surface tracking • Sharper surface detail • Sharper color mixing Grid Only Grid+Particle 12 GTC 2016

Brush-Paint Interaction • Brush emit new pigment particles • Brush carries and pushes existing fluid particles Particle Fluid Brush Sim • Brush pick up color from paint on canvas • Thick paint affect brush motion and shape Grid Fluid 13 GTC 2016

Implementation  CUDA implementation • Both brush and paint liquid in CUDA • CUDA+OpenGL interop. for rendering  Rendering • Ray casting on paint density grid and pigment grid • Screen space particle rendering 14 GTC 2016

Implementation Forward Brush Grid Fluid Forward FLIP k_AdvectPaintDensityGrid Simulation k_ForwardPosition k_Advection_Particles k_FindConvertParticlesToPaintDensity Particles k_VoxelizeBrush Convert inactive Voxelize particles to density Brush Rasterize Hash Brush Pickup Pigment grid k_FilterBrushVoxelization k_BrushPickupPigment Inter-Bristle k_RasterizeParticleTrilinear Particles k_ExtendPigment Collision onto Velocity SolveSelfCollision Find Brush Grid Sample New Particles k_NormalizeRasterization SpawnNewPaticles FindBrushBoundingB ox Bounding Box from Density Grid k_ProjectConstraints Bristle Save current SaveActiveDomain Diffusion Stretching Diffusion and window back to full- k_BrushDistanceMap Create Brush Dissapation k_BrushPaintNodesDistanceToPaintSurface and Bending canvas texture k_UpdateFinalSim PostProcessVelocity Brush Vertices emit Distance Field new Paint Particles k_PaintNodeEmitNewParticlesBasedOnDistanceToPaint Update Brush k_UpdateSorted Attributes for k_MixBrushVelocity Mix with Rendering k_UpdateRendering Brush Resample FLIP Particle k_PIC Velocities Velocity Field k_InterpolatePaintNo Update Brush des Paint Nodes that holds k_UpdatePaintNodesRendering pigment 15 GTC 2016

Performance GTX Titan X  Benchmark sequence: 46 fps, 92K Particles (~2M at maximum) Grid-particle transfer 24% (5.2ms) Bristle-particle transfer (1.6ms) 8% Rendering 9% 24% (2.0ms) 5% Brush simulation Particle-based (1.1ms) liquid simulation (5.2ms) 30% Grid-based liquid simulation (6.5ms) 16 GTC 2016

Performance  Faster stroke requires higher framerate • Faster stroke => more stroke samples per unit time • Maximum distance between stroke sample ~= 3-5 pixels Stroke Sample distance 30px Stroke Sample distance 2px 17 GTC 2016

Optimization 18 GTC 2016

Optimization  Adaptive simulation window (vs fixed simulation window) thrust::minmax_element() Axis-aligned bounding box for brush Active simulation window 46 fps-> 75 fps (63%) For grid fluid simulation 19 GTC 2016

Optimization  Particle insertion Particle Array AtomicInc(Num_Particles) Num_Particles 20 GTC 2016

Optimization  Optimized Particle insertion New Particle Array Marker Array • • Pre-allocate maximum # 1 => a new particle is • No more than 27 particles within one grid cell particle space for each cell added for the element • No more than one particle in one subcell Cell 𝑗 − 1 1 1 0 0 0 0 0 Cell 𝑗 0 1 0 1 1 0 0 0 Cell 𝑗 + 1 1 0 21 GTC 2016

Optimization  Optimized Particle insertion New Particle Array Marker Array Compact New Particle Array • • Pre-allocate maximum # 1 => a new particle is Particle Array particle space for each cell added for the element Cell 𝑗 − 1 1 1 0 0 0 0 0 Cell 𝑗 0 1 0 1 thrust::remove_if() Append 1 0 0 0 Cell 𝑗 + 1 1 0 75 fps-> 92 fps (22%) Shrink List 22 GTC 2016

Kernel Pipeline k_AdvectPaintDensityGrid k_ForwardPosition k_Advection_Particles k_FindConvertParticlesToPaintDensity k_VoxelizeBrush Hash k_FilterBrushVoxelization k_BrushPickupPigment k_RasterizeParticleTrilinear k_ExtendPigment SolveSelfCollision k_NormalizeRasterization SpawnNewPaticles FindBrushBoundingB ox k_ProjectConstraints SaveActiveDomain Diffusion k_BrushPaintNodesDistanceToPaintSurface k_BrushDistanceMap k_UpdateFinalSim PostProcessVelocity k_PaintNodeEmitNewParticlesBasedOnDistanceToPaint k_UpdateSorted k_MixBrushVelocity k_UpdateRendering k_PIC k_InterpolatePaintNo des k_UpdatePaintNodesRendering 23 GTC 2016

Kernel Pipeline k_AdvectPaintDensityGrid k_ForwardPosition k_Advection_Particles k_FindConvertParticlesToPaintDensity k_VoxelizeBrush Hash k_FilterBrushVoxelization k_BrushPickupPigment k_RasterizeParticleTrilinear k_ExtendPigment SolveSelfCollision k_NormalizeRasterization SpawnNewPaticles FindBrushBoundingB ox k_ProjectConstraints SaveActiveDomain Diffusion k_BrushPaintNodesDistanceToPaintSurface k_BrushDistanceMap k_UpdateFinalSim PostProcessVelocity k_PaintNodeEmitNewParticlesBasedOnDistanceToPaint k_UpdateSorted k_MixBrushVelocity k_UpdateRendering k_PIC k_InterpolatePaintNo des k_UpdatePaintNodesRendering Low occupancy, GPU almost idle 24 GTC 2016

Dependencies k_ForwardPosition Hash SolveSelfCollision k_ProjectConstraints SaveActiveDomain k_UpdateFinalSim k_ExtendPigment k_FindConvertParticlesToPaintDensity FindBrushBoundingB k_BrushDistanceMap k_UpdateSorted ox k_PIC k_UpdateRendering k_InterpolatePaintNo k_VoxelizeBrush k_FilterBrushVoxelization des SpawnNewPaticles k_PaintNodeEmitNewParticlesBasedOnDistanceToPaint k_UpdatePaintNodesRendering k_Advection_Particles k_RasterizeParticleTrilinear k_MixBrushVelocity k_AdvectPaintDensityGrid k_BrushPaintNodesDistanceToPaintSurface k_NormalizeRasterization Diffusion k_BrushPickupPigment PostProcessVelocity 25 GTC 2016

Dependencies k_ForwardPosition • Brush # Vertices << Grid Dimension, Particle # Hash • Low occupancy SolveSelfCollision k_ProjectConstraints SaveActiveDomain k_UpdateFinalSim k_ExtendPigment k_FindConvertParticlesToPaintDensity FindBrushBoundingB k_BrushDistanceMap k_BrushDistanceMap k_UpdateSorted ox k_PIC k_UpdateRendering k_InterpolatePaintNo k_VoxelizeBrush k_FilterBrushVoxelization des SpawnNewPaticles k_PaintNodeEmitNewParticlesBasedOnDistanceToPaint k_UpdatePaintNodesRendering k_Advection_Particles k_RasterizeParticleTrilinear k_MixBrushVelocity k_AdvectPaintDensityGrid k_BrushPaintNodesDistanceToPaintSurface k_NormalizeRasterization Diffusion k_BrushPickupPigment • First half of fluid simulation do not depend on brush PostProcessVelocity 26 GTC 2016