Locomotion Animation Gavriel State, Senior Director, Systems - PowerPoint PPT Presentation

Deep Learning for Locomotion Animation Gavriel State, Senior Director, Systems Software March 26, 2018

Deep Learning Animation: PFNN • Breakthrough paper on using motion capture + DL to drive locomotion animation • http://theorangeduck.com/page/phase-functioned-neural-networks-character-control 2

3

Applications Games 4

Applications Crowds 5

Applications Auto Simulation 6 Image from the SYNTHIA dataset

Applications Human/Robot interaction safety 7 Mimus, Madeline Gannon / ATONATON (2016)

Applications Holodeck 8

How does it work? Motion Capture Gather Motion Capture data • Lots of free data available from CMU: http://mocap.cs.cmu.edu/ • Many thanks to Fox VFX Lab for our capture above • 9

How does it work? Metadata labeling Additional data needed: • Gait (running, walking, crouching, etc) • Phase • • Footstep positions 10

How does it work? Terrain Fitting Generate many different • height fields that can fit a given set of character positions More robust than just • capturing the actual height field, since it gives the network more potential data to fit with 11

How does it work? Phase Functioned Neural Network Weights in the network • are different depending on the phase parameter Four sets of weights • trained Mid-cycle weights • calculated by spline interpolation or precomputed (requires custom inferencing code or lots of memory) 12

How does it work? Runtime Inferencing 13

14

PFNN On GPU 15

16

What’s Wrong With This Picture? 17

Here’s a Hint 18

19

DeepLoco: Physics + RL • Another major recent work adds physics and high level control: DeepLoco: Dynamic Locomotion Skills Using Hierarchical Deep Reinforcement Learning Xue Bin Peng (1) Glen Berseth (1) KangKang Yin (2) Michiel van de Panne (1) (1)University of British Columbia (2)National University of Singapore http://www.cs.ubc.ca/~van/papers/2017-TOG-deepLoco/ • 20

How does it work? DeepLoco RL System Simulation engine + RL • • Bullet Physics Engine, rewards • Low level controller Uses phase, like PFNN, but simpler • Activates PD controller • • High Level controller • Generates ‘footstep plan’ based on goals gH Customizable for different tasks • 21

22

Early RL Results DeepLoco-style Reward Function 23

24

Physics + Mocap + RL 25

26

Physics + RL + Uneven Terrain No Mocap Ministry of Silly Walks 27

28

Physics + RL + Uneven Terrain + Mocap Ministry of Getting Closer 29

30

Additional Research Character Interaction 31

32

DEMO: Deep Learning Animation 33

Questions?