scape shape completion scape shape completion and
play

SCAPE: Shape Completion SCAPE: Shape Completion and Animation of - PowerPoint PPT Presentation

SCAPE: Shape Completion SCAPE: Shape Completion and Animation of People and Animation of People By Dragomir Anguelov, Praveen Srinivasan, By Dragomir Anguelov, Praveen Srinivasan, Daphne Koller, Sebastian Thrun, Jim Daphne Koller, Sebastian


  1. SCAPE: Shape Completion SCAPE: Shape Completion and Animation of People and Animation of People By Dragomir Anguelov, Praveen Srinivasan, By Dragomir Anguelov, Praveen Srinivasan, Daphne Koller, Sebastian Thrun, Jim Daphne Koller, Sebastian Thrun, Jim Rodgers, James Davis Rodgers, James Davis From SIGGRAPH 2005 From SIGGRAPH 2005 Presentation for CS468 by Emilio Antú únez nez Presentation for CS468 by Emilio Ant 1 1

  2. Motivation Motivation  It is difficult to get high­resolution body It is difficult to get high­resolution body scans scans  It is even harder at video rates It is even harder at video rates  By building up a human model, you By building up a human model, you could synthesize a high­resolution could synthesize a high­resolution scan from sparse/incomplete data scan from sparse/incomplete data  Accurate model is most easily created Accurate model is most easily created by learning from sample scans by learning from sample scans 2 2

  3. Pre-existing Work in Pre-existing Work in Deformable Human Models I Deformable Human Models I  Deformations described relative to a Deformations described relative to a template shape template shape  Pose deformations given relative to Pose deformations given relative to local joints in an articulated model local joints in an articulated model  Body­shape deformations described Body­shape deformations described using displacement vectors from PCA using displacement vectors from PCA 3 3

  4. Pre-existing Work in Pre-existing Work in Deformable Human Models II Deformable Human Models II  Pose and shape deformations rarely Pose and shape deformations rarely addressed together addressed together  Most similar work by Sumner and Most similar work by Sumner and Popović ć Popovi – Retargets pose deformation to another Retargets pose deformation to another mesh mesh – Does not learn a model Does not learn a model 4 4

  5. Paper Contributions Paper Contributions  Learning an affine deformation model Learning an affine deformation model for both pose and shape for both pose and shape  Shape completion for scan of an Shape completion for scan of an arbitrary human target arbitrary human target  Body shape manipulation for motion Body shape manipulation for motion capture animation capture animation 5 5

  6. Presentation Overview Presentation Overview  Data Acquisition Data Acquisition  Learning the Human Model Learning the Human Model  Applications Applications – Shape Completion Shape Completion – Motion Capture Animation Motion Capture Animation  Limitations Limitations 6 6

  7. Data Format / Assumptions Data Format / Assumptions  Each input model is a deformation of a Each input model is a deformation of a fixed­topology triangle mesh fixed­topology triangle mesh  Models divided into three categories Models divided into three categories – One template model One template model – Template subject in different poses Template subject in different poses – Different people in (roughly) same pose Different people in (roughly) same pose  Articulated skeleton assigned to each Articulated skeleton assigned to each mesh mesh 7 7

  8. Data Acquisition and Data Acquisition and Processing Processing 8 8

  9. Learning the Human Model Learning the Human Model  Pose and shape deformations Pose and shape deformations described per­triangle using linear described per­triangle using linear transformations transformations  Pose transformations learned from Pose transformations learned from template subject in different poses template subject in different poses  Body shape transformations learned Body shape transformations learned by comparing different subjects to by comparing different subjects to template template 9 9

  10. Pose Deformation I Pose Deformation I  Rigid (skeletal) Rigid (skeletal) final triangle deformations are deformations are R l[k] represented separately represented separately from non­rigid ones from non­rigid ones  Transformations are Transformations are given in relative given in relative Q k coordinate system coordinate system O where one of the where one of the corners is fixed at the corners is fixed at the template origin origin triangle 10 10

  11. Pose Deformation II Pose Deformation II  Triangle edges are not Triangle edges are not final triangle forced to be consistent forced to be consistent R l[k]  Final synthesized Final synthesized mesh reduces the mesh reduces the least­squares error least­squares error between mesh points between mesh points Q k O and triangle and triangle deformations deformations template triangle 11 11

  12. Learning Pose Deformation Learning Pose Deformation Model I Model I  Rigid rotation is known from skeleton Rigid rotation is known from skeleton  Non­rigid transformation is Non­rigid transformation is underdefined underdefined  Q matrix is computed by requiring Q matrix is computed by requiring adjacent triangles’ non­rigid adjacent triangles’ non­rigid transformations to be similar transformations to be similar 12 12

  13. Learning Pose Deformation Learning Pose Deformation Model II Model II  Non­rigid deformation modeled as an Non­rigid deformation modeled as an affine function of adjacent joint angles affine function of adjacent joint angles  In practice, some of the degrees of In practice, some of the degrees of freedom are removed for constrained freedom are removed for constrained joings joings 13 13

  14. Pose Deformation Learning Pose Deformation Learning Results Results 14 14

  15. Body-Shape Deformation Body-Shape Deformation  Body shape is modeled as an additional Body shape is modeled as an additional linear transform, S linear transform, S  S is underdetermined (like Q) S is underdetermined (like Q)  Again, solved using a smoothness Again, solved using a smoothness constraint constraint 15 15

  16. Learning the Shape Learning the Shape Deformation Model Deformation Model  The matrix coefficients for all body The matrix coefficients for all body shape transformations are vectorized shape transformations are vectorized  Principal component analysis is used Principal component analysis is used to parameterize the shape transform to parameterize the shape transform vectors vectors 16 16

  17. Shape Deformation Shape Deformation Learning Results Learning Results 17 17

  18. Shape Completion I Shape Completion I  Assuming you know some of the node Assuming you know some of the node positions, estimate the others positions, estimate the others  Must estimate pose and body shape Must estimate pose and body shape  This optimization is highly nonlinear in the This optimization is highly nonlinear in the pose pose  Empirically found that optimizing over all Empirically found that optimizing over all variables at once produces bad results variables at once produces bad results  Instead, SCAPE iterates solving Instead, SCAPE iterates solving 18 18

  19. Shape Completion II Shape Completion II  Empirically found that optimizing over all Empirically found that optimizing over all variables at once produces bad results variables at once produces bad results  Instead, SCAPE iterates, solving each of Instead, SCAPE iterates, solving each of these in order: these in order: – Pose Pose – Mesh estimate Mesh estimate – Body shape Body shape  Results in a “completed” mesh and a Results in a “completed” mesh and a “predicted” mesh “predicted” mesh 19 19

  20. Partial View Completion Partial View Completion  Skeletal and point­correspondences Skeletal and point­correspondences may be off if too much data is missing may be off if too much data is missing  Iterate between the shape completion Iterate between the shape completion algorithm previously described and algorithm previously described and remapping the point correspondences remapping the point correspondences 20 20

  21. Partial View Completion Partial View Completion Results Results 21 21

  22. Motion Capture Animation Motion Capture Animation  Motion capture data provides the pose Motion capture data provides the pose data data  Body shape parameters can be set Body shape parameters can be set arbitrarily arbitrarily  Since markers are generally placed on Since markers are generally placed on body surface (not in the bones), mesh body surface (not in the bones), mesh is constrained to lie in the space of is constrained to lie in the space of body shapes encoded by the model body shapes encoded by the model 22 22

  23. Motion Capture Animation Motion Capture Animation Results Results 23 23

  24. Limitations Limitations  Assumes that pose deformation and Assumes that pose deformation and body shape are mostly independent body shape are mostly independent  Models only pose deformations from Models only pose deformations from skeletal motion skeletal motion 24 24

Recommend


More recommend