Towards a Generative Model of Natural Motion C. Karen Liu - PowerPoint PPT Presentation

Towards a Generative Model of Natural Motion C. Karen Liu University of Southern California

space of motion sequences space of motion sequences space of motion sequences space of motion sequences Manual methods

space of motion sequences space of motion sequences Manual methods Manual methods space of motion sequences space of motion sequences Data-driven methods

space of motion sequences space of motion sequences Data-driven methods Data-driven methods space of motion sequences space of motion sequences Data-driven methods Data-driven methods

space of motion sequences space of motion sequences Physics-based methods space of motion sequences space of motion sequences highly dynamic motion Physics-based methods Physics-based methods

space of motion sequences space of motion sequences highly highly dynamic dynamic motion motion Physics-based methods Physics-based methods space of motion sequences space of motion sequences variational optimization variational optimization highly highly dynamic dynamic motion motion Physics-based methods Physics-based methods

space of motion sequences space of motion sequences variational optimization variational optimization highly highly dynamic dynamic motion motion Physics-based methods Physics-based methods space of motion sequences space of motion sequences variational optimization variational optimization highly highly dynamic dynamic motion motion Physics-based methods Physics-based methods

space of motion sequences space of motion sequences variational optimization variational optimization highly highly dynamic dynamic motion motion Physics-based methods Physics-based methods space of motion sequences space of motion sequences variational optimization highly highly low energy dynamic dynamic motion motion motion Physics-based methods Generative model

space of motion sequences space of motion sequences highly highly low energy low energy dynamic dynamic motion motion motion motion Generative model Generative model space of motion sequences space of motion sequences highly highly low energy low energy dynamic dynamic motion motion motion motion Generative model Generative model

space of motion sequences highly highly low energy dynamic dynamic motion motion motion Generative model

space of motion sequences space of motion sequences highly highly low energy dynamic dynamic motion motion motion space of motion sequences space of motion sequences highly highly low energy low energy dynamic dynamic motion motion motion motion

space of motion sequences space of motion sequences highly highly low energy low energy dynamic dynamic motion motion motion motion space of motion sequences space of motion sequences highly highly low energy low energy dynamic dynamic motion motion motion motion

space of motion sequences low energy highly low energy dynamic motion motion motion input motion

input motion input constraints input motion input constraints output motion Physical model

muscle force usage � � Q 2 Q 2 E ( X ) = E ( X ) = m j m j j j motion muscle force usage motion muscle force usage � � Q 2 Q 2 E ( X ) = E ( X ) = m j m j j j

There is a distinct preference for using specific joints rather than others, due to variations in join strength, stability, and other factors [Full et al. 2002]. motion muscle force usage motion muscle force usage � � Q 2 Q 2 E ( X ) = E ( X ) = m j m j j j � � α j Q 2 α j Q 2 E ( X ) = E ( X ) = m j m j j j

� � α j Q 2 α j Q 2 E ( X ) = E ( X ) = m j m j j j Lagrangian dynamics

Lagrangian Lagrangian dynamics dynamics d ∂ T q − ∂ T d ∂ T q − ∂ T ∂ q = Q int + Q ext ∂ q = Q int + Q ext dt ∂ ˙ dt ∂ ˙ Lagrangian Lagrangian dynamics dynamics d ∂ T q − ∂ T d ∂ T q − ∂ T � �� � ∂ q = Q int + Q ext ∂ q = Q m + Q p + Q ext dt ∂ ˙ dt ∂ ˙

Lagrangian Hooke’s law dynamics d ∂ T q − ∂ T � �� � ∂ q = Q m + Q p + Q ext dt ∂ ˙ Hooke’s law Hooke’s law Biological systems use passive elements, such as tendons and ligaments, to store and release energy, thereby reducing total power consumption [Alexandar 1998].

Hooke’s law Hooke’s law Biological systems use passive elements, such as tendons and ligaments, to store and release energy, thereby reducing total power consumption [Alexandar 1998]. Animals vary stiffness of their joints when performing different tasks [Farley and Morgenroth 1999]. Lagrangian Hooke’s law dynamics Q p = − k s ( q − ¯ q ) − k d ˙ q d ∂ T q − ∂ T � �� � ∂ q = Q m + Q p + Q ext dt ∂ ˙

Lagrangian Lagrangian dynamics dynamics Q g Q g Q c d ∂ T q − ∂ T d ∂ T q − ∂ T � �� � � �� � � �� � � �� � ∂ q = Q m + Q p + Q g ∂ q = Q m + Q p + Q g + Q c dt ∂ ˙ dt ∂ ˙ Lagrangian Lagrangian dynamics dynamics Q g Q g Q s Q s Q c Q c d ∂ T q − ∂ T Q m = d ∂ T q − ∂ T � �� � � �� � ∂ q = Q m + Q p + Q g + Q c + Q s ∂ q − Q p − Q g − Q c − Q s dt ∂ ˙ dt ∂ ˙

Hooke’s law Hooke’s law Lagrangian Lagrangian dynamics dynamics Q p = − k s ( q − ¯ q ) − k d ˙ Q p = − k s ( q − ¯ q ) − k d ˙ q q Q m = d ∂ T q − ∂ T Q m = d ∂ T q − ∂ T ∂ q − Q p − Q g − Q c − Q s ∂ q − Q p − Q g − Q c − Q s ∂ ˙ ∂ ˙ dt dt � � α j Q 2 α j Q 2 E ( X ) = E ( X ) = α j m j m j j j Hooke’s law Hooke’s law Lagrangian Lagrangian dynamics dynamics Q p = − k s ( q − ¯ q ) − k d ˙ Q p = − k s ( q − ¯ q ) − k d ˙ k s k d q k s k d q ¯ ¯ q q Q m = d ∂ T q − ∂ T Q m = d ∂ T q − ∂ T ∂ q − Q p − Q g − Q c − Q s ∂ q − Q p − Q g − Q c − Q s ∂ ˙ ∂ ˙ dt dt α k s k d ¯ θ = { , } q , , � � α j Q 2 α j Q 2 E ( X ) = E ( X ) = α j α j m j m j j j

Hooke’s law Lagrangian dynamics Q p = − k s ( q − ¯ q ) − k d ˙ k s k d q ¯ q Q m = d ∂ T q − ∂ T ∂ q − Q p − Q g − Q c − Q s ∂ ˙ dt α k s k d ¯ θ = { , } q , , � α j Q 2 E ( X ) = preference of muscle usage α j m j style { j stiffness of passive elements emotional state emotional state individual

preference of muscle usage style α k s k d ¯ { θ = { , } q , , stiffness of passive elements emotional state individual activity preference of muscle usage preference of muscle usage style style α k s k d ¯ { α k s k d ¯ { θ = { , } θ = { , } q q , , stiffness of passive elements , , stiffness of passive elements X ∗ = argmin E ( X ; θ ) X ∗ = argmin E ( X ; θ ) X ∈ C X ∈ C

preference of muscle usage preference of muscle usage style style α k s k d ¯ { α k s k d ¯ { θ = { , } θ = { , } q q , , stiffness of passive elements , , stiffness of passive elements X ∗ = argmin E ( X ; θ ) X ∗ = argmin E ( X ; θ ) X ∈ C X ∈ C preference of muscle usage preference of muscle usage style style α k s k d ¯ { α k s k d ¯ { θ = { , } θ = { , } q q , , stiffness of passive elements , , stiffness of passive elements X ∗ = argmin E ( X ; θ ) X ∗ = argmin E ( X ; θ ) X ∈ C X ∈ C

preference of muscle usage preference of muscle usage style style α k s k d ¯ { α k s k d ¯ { θ = { , } θ = { , } q q , , stiffness of passive elements , , stiffness of passive elements X ∗ = argmin E ( X ; θ ) X ∗ = argmin E ( X ; θ ) X ∈ C X ∈ C E ( X , ) E ( X , ) θ θ X X X ∗ preference of muscle usage preference of muscle usage style style α k s k d ¯ { α k s k d ¯ { θ = { , } θ = { , } q q , , stiffness of passive elements , , stiffness of passive elements X ∗ = argmin E ( X ; θ ) X ∗ = argmin E ( X ; θ ) X ∈ C X ∈ C E ( X , ) E ( X , ) θ θ X X X ∗ X ∗ X ∗ X ∗ X ∗

preference of muscle usage preference of muscle usage style style α k s k d ¯ { α k s k d ¯ { θ = { , } θ = { , } q q , , stiffness of passive elements , , stiffness of passive elements X ∗ = argmin E ( X ; θ ) X ∗ = argmin E ( X ; θ ) X ∈ C X ∈ C has about 150 degrees of freedom θ Nonlinear inverse optimization Nonlinear inverse optimization Given target motion and constraints , X T C

Nonlinear inverse optimization Nonlinear inverse optimization Given target motion and constraints , Given target motion and constraints , X T X T C C determine style parameters θ Nonlinear inverse optimization Nonlinear inverse optimization Given target motion and constraints , Given target motion and constraints , X T X T C C determine style parameters θ determine style parameters θ E ( X ; θ ) X X T

Nonlinear inverse optimization Nonlinear inverse optimization Given target motion and constraints , Given target motion and constraints , X T X T C C determine style parameters θ determine style parameters θ E ( X ; θ ) E ( X ; θ ) X X X T X T What does not work? What does not work? Least square difference