real time facial animation
play

Real-time Facial Animation Hao Li Mark Pauly ILM EPFL High-End - PowerPoint PPT Presentation

Mar 14, 2024 •157 likes •1.67k views

Real-time Facial Animation Hao Li Mark Pauly ILM EPFL High-End 3D Scanning High-End 3D Scanning Low-Cost Passive Scanning (AGI soft) stereo pair Low-Cost Passive Scanning (AGI soft) stereo pair 3D scan Low-Cost Active Scanning +

  1. Real-time Facial Animation Hao Li Mark Pauly ILM EPFL

  2. High-End 3D Scanning

  3. High-End 3D Scanning

  4. Low-Cost Passive Scanning (AGI soft) stereo pair

  5. Low-Cost Passive Scanning (AGI soft) stereo pair 3D scan

  6. Low-Cost Active Scanning + Temporal Upsampling Microsoft Kinect & Kinect Fusion

  7. Rigging & Animation

  8. Rigging & Animation

  9. Typical Facial Animation Workflow in Industry

  10. Typical Facial Animation Workflow in Industry 3D Scanning

  11. Typical Facial Animation Workflow in Industry Modeling + 3D Scanning Fitting

  12. Typical Facial Animation Workflow in Industry Modeling + Light-weight 3D Scanning Fitting Rigging

  13. Typical Facial Animation Workflow in Industry Modeling + Light-weight Motion 3D Scanning Fitting Rigging Capture

  14. Typical Facial Animation Workflow in Industry Modeling + Light-weight Motion Cleanup & 3D Scanning Fitting Rigging Capture Key-Framing

  15. Typical Facial Animation Workflow in Industry Modeling + Light-weight Motion Cleanup & 3D Scanning Fitting Rigging Capture Key-Framing Modeling

  16. Typical Facial Animation Workflow in Industry Modeling + Light-weight Motion Cleanup & 3D Scanning Fitting Rigging Capture Key-Framing Complex Modeling Rigging

  17. Typical Facial Animation Workflow in Industry Modeling + Light-weight Motion Cleanup & 3D Scanning Fitting Rigging Capture Key-Framing Complex Modeling Retargeting Rigging

  18. Typical Facial Animation Workflow in Industry Modeling + Light-weight Motion Cleanup & 3D Scanning Fitting Rigging Capture Key-Framing Complex Modeling Retargeting Rigging

  19. Typical Facial Animation Workflow in Industry Modeling + Light-weight Motion Cleanup & 3D Scanning Fitting Rigging Capture Key-Framing Complex Key-Framing + Modeling Retargeting Rigging Proc.+Sim.

  20. Markerless Facial Capture

  21. 3D range sensor

  22. 3D range sensor

  23. 3D range sensor Motion can be Captured at the Same Resolution as the Geometry

  24. USC ICT Light Stage 5

  25. USC ICT Light Stage 5

  26. Goal

  27. Goal

  28. Template-Based Tracking

  29. Template-Based Tracking

  30. Template-Based Tracking analyze deformation

  31. Template-Based Tracking analyze deformation

  32. Template-Based Tracking transfer deformation

  33. Template-Based Tracking transfer deformation

  34. Template-Based Tracking transfer deformation

  35. Template-Based Tracking transfer deformation

  36. Correspondences Problem

  37. Correspondences Problem

  38. Correspondences Problem

  39. Correspondences Problem

  40. Non-Rigid Registration

  41. Pair of 3D Scans

  42. Pair of 3D Scans source

  43. Pair of 3D Scans target source

  44. Correspondences are Lost

  45. Correspondences are Lost

  46. Correspondences are Lost ?

  47. Overlapping Regions are Lost

  48. Overlapping Regions are Lost overlapping regions

  49. Overlapping Regions are Lost missing data overlapping regions

  50. Overlapping Regions are Lost

  51. Overlapping Regions are Lost

  52. Non-Rigid Registration

  53. Non-Rigid Registration

  54. Three Ingredients

  55. Three Ingredients source

  56. Three Ingredients source target

  57. Three Ingredients source target

  58. Three Ingredients source target detect overlap

  59. Three Ingredients source target detect overlap

  60. Three Ingredients source target detect correspond overlap

  61. Three Ingredients source target detect correspond overlap

  62. Three Ingredients source target detect correspond deform overlap

  63. Three Ingredients registration target detect correspond deform overlap

  64. Challenges detect correspond deform overlap

  65. Challenges detect correspond deform overlap

  66. Challenges detect correspond deform overlap

  67. Challenges deformation detect correspond deform overlap

  68. Challenges detect correspond deform overlap

  69. Challenges detect correspond deform overlap

  70. Challenges ambiguity detect correspond deform overlap

  71. Challenges detect correspond deform overlap

  72. Challenges detect correspond deform overlap

  73. Challenges detect correspond deform overlap

  74. Challenges detect correspond deform overlap

  75. Challenges ? detect correspond deform overlap

  76. Challenges detect correspond deform overlap

  77. Challenges correspond detect deform overlap

  78. Observation correspond detect deform overlap

  79. Observation correspond detect deform overlap

  80. Observation correspond helps detect deform overlap

  81. Observation correspond helps helps detect deform overlap

  82. Observation correspond detect deform overlap

  83. Observation correspond detect deform overlap

  84. Observation correspond detect deform overlap global optimization via local refinement

  85. Iterative Global Optimization correspond detect deform overlap

  86. Iterative Global Optimization correspond detect overlap deform

  87. Iterative Global Optimization correspond detect overlap deform

  88. Iterative Global Optimization correspond detect overlap deform

  89. Iterative Global Optimization correspond detect overlap deform

  90. Iterative Global Optimization correspond detect overlap deform

  91. Iterative Global Optimization closest point detect overlap deform

  92. Iterative Global Optimization closest point detect overlap deform

  93. Iterative Global Optimization closest point pruning deform

  94. Iterative Global Optimization closest point pruning deform

  95. Iterative Global Optimization closest point pruning deform

  96. Iterative Global Optimization closest point pruning deform

  97. Iterative Global Optimization closest point pruning deform

  98. Iterative Global Optimization closest point pruning global optimization

  99. Iterative Global Optimization closest point pruning global optimization

  100. Iterative Global Optimization closest point pruning global optimization converges?

Recommend

Real-time Performance-Based Facial Animation Mark Pauly Eurographics 2012, Cagliari, Italy

Real-time Performance-Based Facial Animation Mark Pauly Eurographics 2012, Cagliari, Italy

Real-time Performance-Based Facial Animation Mark Pauly Eurographics 2012, Cagliari, Italy Introduction 2 Eurographics 2012, Cagliari, Italy Overview Facial Tracking using Blendshape Model rigid transformation + blendshape weights per

542 views • 19 slides
Performance-Driven Facial Animation Performance-based Facial Animation Creating an animation

Performance-Driven Facial Animation Performance-based Facial Animation Creating an animation

Performance-Driven Facial Animation Performance-based Facial Animation Creating an animation of a realistic and expressive human face is one of greatest challenges in computer graphics. The human face is an extremely complex

492 views • 28 slides
Real-time Animation and Real-time Animation and Rendering of Ocean Whitecaps Rendering of Ocean

Real-time Animation and Real-time Animation and Rendering of Ocean Whitecaps Rendering of Ocean

Real-time Animation and Real-time Animation and Rendering of Ocean Whitecaps Rendering of Ocean Whitecaps Jonathan Dupuy, Eric Bruneton INRIA Grenoble Rhne-Alpes, Universit de Grenoble et CNRS, Laboratoire Jean Kuntzmann Universit de

558 views • 26 slides
Real-time 3D Eyelids Tracking From Semantic Edges Quan Wen, Feng Xu, Ming Lu, Jun-Hai Yong

Real-time 3D Eyelids Tracking From Semantic Edges Quan Wen, Feng Xu, Ming Lu, Jun-Hai Yong

Real-time 3D Eyelids Tracking From Semantic Edges Quan Wen, Feng Xu, Ming Lu, Jun-Hai Yong Tsinghua University Background Facial capture and animation is crucial in many applications Face capture in computer games Face animation in movies

944 views • 79 slides
Data-Driven Animation Full-body animation Skin deformation Facial animation Motion

Data-Driven Animation Full-body animation Skin deformation Facial animation Motion

Data-Driven Animation Full-body animation Skin deformation Facial animation Motion Representation A pose is represented by root translation, root orientation, and joint angles q = ( q 1 , , q n ) Motion consists of a

368 views • 24 slides
FMP Pitch Introduction The path I am keen to work with is an action animation and facial

FMP Pitch Introduction The path I am keen to work with is an action animation and facial

Joe Agilada M00486462 FMP Pitch Introduction The path I am keen to work with is an action animation and facial expression animation such as lip syncing. But for my fjnal major project I want to create a VR narrative animation where character

423 views • 13 slides
Facial Expressions & Rigging CSE169: Computer Animation Instructor: Steve Rotenberg UCSD,

Facial Expressions & Rigging CSE169: Computer Animation Instructor: Steve Rotenberg UCSD,

Facial Expressions & Rigging CSE169: Computer Animation Instructor: Steve Rotenberg UCSD, Spring 2016 Facial Muscles Universal Expression Groups Sadness Anger Happiness Fear Disgust Surprise FACS Facial Action

1.03k views • 59 slides
Facial Expressions & Rigging CSE169: Computer Animation Instructor: Steve Rotenberg UCSD,

Facial Expressions & Rigging CSE169: Computer Animation Instructor: Steve Rotenberg UCSD,

Facial Expressions & Rigging CSE169: Computer Animation Instructor: Steve Rotenberg UCSD, Winter 2020 Facial Muscles Universal Expression Groups Sadness Anger Happiness Fear Disgust Surprise FACS Facial

1.09k views • 51 slides
Chapter 8 - Animation Animation before the time of 3DCG Animation techniques

Chapter 8 - Animation Animation before the time of 3DCG Animation techniques

Chapter 8 - Animation Animation before the time of 3DCG Animation techniques keyframing bone animation & motion capture simulation Animation principles LMU Mnchen Medieninformatik Heinrich Hussmann

812 views • 47 slides
Real Time Facial Expression Recognition using Eigen Faces - By Aadesh M Bagmar (U12 CO 092)

Real Time Facial Expression Recognition using Eigen Faces - By Aadesh M Bagmar (U12 CO 092)

Summer Intern Project on Real Time Facial Expression Recognition using Eigen Faces - By Aadesh M Bagmar (U12 CO 092) Project Guide: Dr. M.A.Zaveri Computer Engineering Department Brief The project aims at developing a robust system for real

206 views • 18 slides
WE MAKE DIGITAL HUMANS CONTENTS SCANNING 3D EXTRAS - FACIAL SCANNING - HAIR + CLOTH -

WE MAKE DIGITAL HUMANS CONTENTS SCANNING 3D EXTRAS - FACIAL SCANNING - HAIR + CLOTH -

WE MAKE DIGITAL HUMANS CONTENTS SCANNING 3D EXTRAS - FACIAL SCANNING - HAIR + CLOTH - BODY SCANNING - REAL-TIME INTEGRATION MODELLING/TEXTURING - LIGHTING + RENDERING - CAMERA ANIMATION - MODELLING - HEAD TRACKING -

728 views • 42 slides
animation 1 animation shape specification as a function of time 2 animation representation

animation 1 animation shape specification as a function of time 2 animation representation

animation 1 animation shape specification as a function of time 2 animation representation many ways to represent changes with time intent artistic motion physically-plausible motion efficiency typically not a major problem control most

756 views • 74 slides
Animation structure and content of the internal of changes over time? representation. Why?

Animation structure and content of the internal of changes over time? representation. Why?

Animation Animation: can it facilitate? Congruence Principle A useful graphic is congruent to the Does animation help the understanding Is animation useful? Animation structure and content of the internal of changes over time?

423 views • 4 slides
Realistic Modeling for Realistic Modeling for Facial Animation Facial Animation Yuencheng Lee,

Realistic Modeling for Realistic Modeling for Facial Animation Facial Animation Yuencheng Lee,

Realistic Modeling for Realistic Modeling for Facial Animation Facial Animation Yuencheng Lee, Lee, Demetri Demetri Yuencheng Terzopoulos, and Keith Waters. , and Keith Waters. Terzopoulos Overview Overview Desire Desire A

349 views • 13 slides
Annotated Facial Landmarks in the Wild A Large-scale, Real-world Database for Facial Landmark

Annotated Facial Landmarks in the Wild A Large-scale, Real-world Database for Facial Landmark

Institute for Computer Graphics and Vision, Graz University of Technology, Austria Annotated Facial Landmarks in the Wild A Large-scale, Real-world Database for Facial Landmark Localization Martin Kstinger, Paul Wohlhart, Peter M. Roth,

310 views • 14 slides
Animation Presented by Sancho McCann Animation Is animation useful? Why?

Animation Presented by Sancho McCann Animation Is animation useful? Why?

Animation Presented by Sancho McCann Animation Is animation useful? Why? Principles of animation Principles applied Animation: can it facilitate? Does animation help the understanding of changes over time? A picture

4.12k views • 49 slides
Review CSE169: Computer Animation Instructor: Steve Rotenberg UCSD, Spring 2016 Character

Review CSE169: Computer Animation Instructor: Steve Rotenberg UCSD, Spring 2016 Character

Review CSE169: Computer Animation Instructor: Steve Rotenberg UCSD, Spring 2016 Character Animation Rigging Skeleton Skinning Shape interpolation (& facial expressions) DOF mapping Animation System Channels

1.38k views • 117 slides
CS324e - Elements of Graphics and Visualization Animation in Java3D Adding Animation

CS324e - Elements of Graphics and Visualization Animation in Java3D Adding Animation

CS324e - Elements of Graphics and Visualization Animation in Java3D Adding Animation Animation in Java2D achieved by changing the position (or some other attribute) of the graphic primitive over time Animation framework from FRC

666 views • 29 slides
Network Layout Ma Maneesh Agrawala CS 448B: Visualization Fall 2020 1 Last Time: Animation 2

Network Layout Ma Maneesh Agrawala CS 448B: Visualization Fall 2020 1 Last Time: Animation 2

Network Layout Ma Maneesh Agrawala CS 448B: Visualization Fall 2020 1 Last Time: Animation 2 1 Implementing Animation 3 Animation Approaches Frame-based Animation Redraw scene at regular interval (e.g., 16ms) Developer defines the

837 views • 66 slides
Modeling Continuous Shape Change for Facial Animation Julian Faraway Department of Statistics

Modeling Continuous Shape Change for Facial Animation Julian Faraway Department of Statistics

Modeling Continuous Shape Change for Facial Animation Julian Faraway Department of Statistics University of Michigan Cleft Lip and Palate A B C Sequence of surgeries is recommended. Cleft lip surgery at 3 months and palate surgery around 1

338 views • 20 slides
Rishabh Battulwar Research and Development at Digital Domain VFx pipelines Fluids Hair Smoke

Rishabh Battulwar Research and Development at Digital Domain VFx pipelines Fluids Hair Smoke

Data-Driven Future in Visual Effects Rishabh Battulwar Research and Development at Digital Domain VFx pipelines Fluids Hair Smoke Fire Rigid Body Facial Animation What s important? Natural Facial Shapes Realistic Appearance Facial

396 views • 36 slides
DESIGN OF A REAL-TIME ANIMATION PROGRAM DEMO MODEL-VIEW-CONTROLLER DESIGN canonical.cpp

DESIGN OF A REAL-TIME ANIMATION PROGRAM DEMO MODEL-VIEW-CONTROLLER DESIGN canonical.cpp

DONALD HOUSE: CPSC 8170, FALL 2018 DESIGN OF A REAL-TIME ANIMATION PROGRAM DEMO MODEL-VIEW-CONTROLLER DESIGN canonical.cpp The main program Controller canonical.cpp acts as the controller, directing Model and View actions parameter

399 views • 21 slides
Final Demos on Thursday! 1. Robotic Arm Controller for Diagnostics 2. PiCar 3. Facial

Final Demos on Thursday! 1. Robotic Arm Controller for Diagnostics 2. PiCar 3. Facial

Final Demos on Thursday! 1. Robotic Arm Controller for Diagnostics 2. PiCar 3. Facial Recognition Based Attendance 4. Interactive Sign Language Training System 5. WashU Course Buddy IoT 6. Smart Soft Real-Time Security Camera 7.

932 views • 57 slides
Real- Real -Time Systems Time Systems Real- -Time Systems Time Systems Real

Real- Real -Time Systems Time Systems Real- -Time Systems Time Systems Real

EDA222/DIT160 Real-Time Systems, Chalmers/GU, 2008/2009 Lecture #15 Updated 2009-03-03 Dependable Distributed Dependable Distributed Real- Real -Time Systems Time Systems Real- -Time Systems Time Systems Real Aircraft/automotive

501 views • 9 slides

More recommend