poselets body part detectors trained using 3d human pose
play

Poselets: Body Part Detectors Trained Using 3D Human Pose - PowerPoint PPT Presentation

Jun 14, 2023 •196 likes •733 views

Poselets: Body Part Detectors Trained Using 3D Human Pose Annotations Lubomir Bourdev and Jitendra Malik Experiments Presented by Randall Smith Friday, November 2, 12 1 Outline Introduction Dataset Overview Annotations

  1. Poselets: Body Part Detectors Trained Using 3D Human Pose Annotations Lubomir Bourdev and Jitendra Malik Experiments Presented by Randall Smith Friday, November 2, 12 1

  2. Outline • Introduction • Dataset • Overview • Annotations • Distance Function • Segmentation • Experiments • Conclusion Friday, November 2, 12 2

  3. Dataset • Humans in 3D (H3D) • 2480 annotations • (1500 train / 500 test / 240 validate) • Java3D annotation tool Friday, November 2, 12 3

  4. Dataset Friday, November 2, 12 4

  5. Overview : Training Random Seed Select Nearby Train SVMs Prune Patches Patches Residual Error Friday, November 2, 12 5

  6. Overview : Detection Run All Mean Shift Compute Max Margin Poselets Cluster Activations Hough Friday, November 2, 12 6

  7. Annotations • Bounding box placed over annotated figure Friday, November 2, 12 7

  8. Annotations • Live Demo Friday, November 2, 12 8

  9. Annotations • Live Demo Friday, November 2, 12 9

  10. Annotations : Skeleton • Annotated skeleton Friday, November 2, 12 10

  11. Annotations : Keypoints • 20 manually annotated keypoints • 15 manually annotated segments Friday, November 2, 12 11

  12. Annotations : Query Mean Examples Distance 0 0 0 0.0001 0.0001 0.0002 0.0002 0.0003 0.0005 0.0005 • Query at green box Friday, November 2, 12 12

  13. Annotations : Query Mean Examples Distance 0 0.0017 0.0020 0.0020 0.0028 0.0031 0.0032 0.0035 0.0041 0.0043 Friday, November 2, 12 13

  14. Annotations : Query Mean Examples Distance 0 0.0019 0.0079 0.0093 0.0096 0.0104 0.0134 0.0139 0.0154 0.0169 Friday, November 2, 12 14

  15. Annotations : Query Mean Examples Distance 0 0.0067 0.0086 0.0167 0.0176 0.0178 0.0180 0.0183 0.0186 0.0198 Friday, November 2, 12 15

  16. Distance Function • Paper: computes a weighted sum of Euclidean distances with additive penalty. • Implementation: Procrustes distance plus penalty. • What is the Procrustes distance? Friday, November 2, 12 16

  17. Distance Function D proc ( x 1 , x 2 ) = min s,R,t k x 1 � ( sRx 2 + t ) k • Scale so that RMS is 1.0, translate to origin, and solve for rotation matrix R. • Non visible key points ignored Friday, November 2, 12 17

  18. Distance Function D ( x s , x r ) = D proc ( x s , s r ) + Penalty • Need to compute linear least squares / SVD to solve. • Is this very expensive? Friday, November 2, 12 18

  19. Distance Function • Live Demo: 2D Toy Example Friday, November 2, 12 19

  20. Segments : UpperClothes Friday, November 2, 12 20

  21. Segments : LowerClothes Friday, November 2, 12 21

  22. Segments : Faces Friday, November 2, 12 22

  23. Detection • A simple, occlusion free test Friday, November 2, 12 23

  24. Detection Task Time Features 0.69s Detect Poselets 0.56s Score 0.82s Cluster 0.61s Localize 0.11s Total 2.49s Friday, November 2, 12 24

  25. Detection : Example • Score: 14.10. How did the clusters vote? Friday, November 2, 12 25

  26. Detection : Votes • Inspect top hits • Inspect bottom hits Friday, November 2, 12 26

  27. Detection : Best Votes Friday, November 2, 12 27

  28. Detection : Best Votes Friday, November 2, 12 28

  29. Detection : Best Votes Friday, November 2, 12 29

  30. Detection : Best Votes Friday, November 2, 12 30

  31. Detection : Worst Votes Friday, November 2, 12 31

  32. Detection : Worst Votes Friday, November 2, 12 32

  33. Detection : Worst Votes Friday, November 2, 12 33

  34. Detection : Worst Votes Friday, November 2, 12 34

  35. Detection : Tests • Some samples from PASCAL VOC2007 • With varying degrees of occlusion • Comparison with Discriminatively Trained Deformable Part Models (DPM) • Some pictures taken from my iPhone 4S • Increasingly difficult in terms of occlusion Friday, November 2, 12 35

  36. Detection : Tests • PASCAL VOC2007 Friday, November 2, 12 36

  37. Detection : Tests • Some more difficult occlusion cases Friday, November 2, 12 37

  38. Detection : Comparison • Scores: 11.50 and 2.03. • DPM failed. Friday, November 2, 12 38

  39. Detection : Comparison • Score: 5.21. 31 poselet clusters contributed. • DPM HOG parts and bounding box shown. Friday, November 2, 12 39

  40. Detection : Comparison • Scores 12.48, 9.67, and 5.40. • DPM failed. Friday, November 2, 12 40

  41. Detection : Comparison • Score 12.01. • HOG parts and bounding box shown. Friday, November 2, 12 41

  42. Detection : Comparison • Score: 5.21. • HOG parts and bounding box shown. Friday, November 2, 12 42

  43. Detection : Comparison • Both fail. Friday, November 2, 12 43

  44. Detection : Comparison • When both succeed, DPM seems to get better bounding boxes. • The poselet algorithm always tries to get the best bounding box it can. • DPM has no way of degrading gracefully. Friday, November 2, 12 44

  45. Detection : Occlusion • Score: 22.4. 54 poselet clusters contributed. Friday, November 2, 12 45

  46. Detection : Occlusion • Score: 0.29. 3 poselet clusters contributed. Friday, November 2, 12 46

  47. Detection : Occlusion • Score: 0.38. 2 poselet clusters contributed. Friday, November 2, 12 47

  48. Detection : Occlusion • Score: 0.27. 1 poselet cluster contributed. Friday, November 2, 12 48

  49. Detection : Occlusion • Score: 0.21. 2 poselet clusters contributed. Friday, November 2, 12 49

  50. Detection : Occlusion • DPM fails on all of these. • Poselets do pretty poorly, but it still computes a bounding box. • Poselets have the chance of getting it right. Friday, November 2, 12 50

  51. Conclusions • Poselets are intuitive to find in an image. • If a body part is exposed, a poselet might match it. • Poselet ranking and scoring can be understood in an intuitive way. • Can handle some occlusion • Will always try to compute a bounding box. Friday, November 2, 12 51

  52. Conclusions • Sometimes poselet activations can be misleading. • Sometimes, some poselets should have higher scores than others. • This is sort of like getting the right answer for the wrong reasons. • The dataset is very labor intensive. Friday, November 2, 12 52

Recommend

Poselets: Body Part Detectors Trained Using 3D Human Pose Annotations LUBOMIR BOURDEV AND

Poselets: Body Part Detectors Trained Using 3D Human Pose Annotations LUBOMIR BOURDEV AND

Poselets: Body Part Detectors Trained Using 3D Human Pose Annotations LUBOMIR BOURDEV AND JITENDRA MALIK Outline H3D dataset Pipeline Analysis of Poselets fired Selective parts torso, legs and face Other cases Clutter, Rotation and

765 views • 37 slides
Human Pose Search using Deep Poselets Nataraj Jammalamadaka * Andrew Zisserman C. V. Jawahar *

Human Pose Search using Deep Poselets Nataraj Jammalamadaka * Andrew Zisserman C. V. Jawahar *

Human Pose Search using Deep Poselets Nataraj Jammalamadaka * Andrew Zisserman C. V. Jawahar * * CVIT, IIIT Hyderabad, India Visual Geometry Group, Department of Engineering, IIIT Hyderabad University of Oxford Human Pose: Gesture and

553 views • 33 slides
Learning Human Pose from Unaligned Data through Image Translation Tomas Jakab Ankush Gupta

Learning Human Pose from Unaligned Data through Image Translation Tomas Jakab Ankush Gupta

Learning Human Pose from Unaligned Data through Image Translation Tomas Jakab Ankush Gupta Andrea Vedaldi Hakan Bilen Presented by Triantafyllos Afouras Goal Learn human-body landmark detectors from unlabelled videos and unaligned

402 views • 27 slides
Articulated body dynamics Beyond human models How would you represent a pose? Quadraped animals

Articulated body dynamics Beyond human models How would you represent a pose? Quadraped animals

Articulated rigid bodies Articulated body dynamics Beyond human models How would you represent a pose? Quadraped animals Wavy hair Animal fur Plants Maximal vs. reduced coordinates How are things connected? Maximal coordinates Reduced

265 views • 10 slides
CEF IPA arm 11 102 the long part at each side of the human body, ending in a hand noun My

CEF IPA arm 11 102 the long part at each side of the human body, ending in a hand noun My

WORD UNIT NO PAGE DEFINITION PoS EXAMPLE CEF IPA arm 11 102 the long part at each side of the human body, ending in a hand noun My arms ache from carrying this bag. m one of the parts of the body of a human or animal that is used

172 views • 3 slides
Human Pose Estimation by Yannic Jnike - 04.11.2019 https://www.youtube.com/watch?v=mxKlUO_tjcg

Human Pose Estimation by Yannic Jnike - 04.11.2019 https://www.youtube.com/watch?v=mxKlUO_tjcg

Human Pose Estimation by Yannic Jnike - 04.11.2019 https://www.youtube.com/watch?v=mxKlUO_tjcg 1 Human Pose Estimation 1. What is Human Pose Estimation 2. OpenPose Pipeline 3. Bottom Up or Top Down Approach 2 What is Human Pose

3.09k views • 33 slides
Deep neural nets for human pose estimation in videos Tomas Pfister, James Charles, Andrew

Deep neural nets for human pose estimation in videos Tomas Pfister, James Charles, Andrew

Deep neural nets for human pose estimation in videos Tomas Pfister, James Charles, Andrew Zisserman Department of Engineering Science University of Oxford http://www.robots.ox.ac.uk/~vgg Aim: Estimate 2D upper body joint positions (wrist,

519 views • 28 slides
Human Pose Recovery And Gesture Recognition CS365 : Artificial Intelligence Khandesh

Human Pose Recovery And Gesture Recognition CS365 : Artificial Intelligence Khandesh

Human Pose Recovery And Gesture Recognition CS365 : Artificial Intelligence Khandesh Bhange(11196) And Piyush Kumar(11496) Problem Statement There are two parts of this project. 1 st Human Pose Recovery We have to label 14 limbs

389 views • 15 slides
7a A&P: Introduction to the Human Body - Body Compass 7a A&P: Introduction to the Human

7a A&P: Introduction to the Human Body - Body Compass 7a A&P: Introduction to the Human

7a A&P: Introduction to the Human Body - Body Compass 7a A&P: Introduction to the Human Body - Body Compass Class Outline 5 minutes Attendance, Breath of Arrival, and Reminders 10 minutes Lecture: AOIs of the

399 views • 38 slides
Human Pose Es,ma,on Greg Mori CMPT 888 Problem Human

Human Pose Es,ma,on Greg Mori CMPT 888 Problem Human

Human Pose Es,ma,on Greg Mori CMPT 888 Problem Human Figures in S,ll Images Detec,on of humans is possible for stereotypical poses Standing

503 views • 46 slides
Chirality Nets for Human Pose Regression Raymond A. Yeh*, Yuan-Ting Hu*, Alexander G. Schwing

Chirality Nets for Human Pose Regression Raymond A. Yeh*, Yuan-Ting Hu*, Alexander G. Schwing

Chirality Nets for Human Pose Regression Raymond A. Yeh*, Yuan-Ting Hu*, Alexander G. Schwing University of Illinois at Urbana-Champaign December 14, 2019 Yeh et al. 1 NeurIPS2019 Motivation Human Pose Regression Prior works: Concatenation

1.21k views • 16 slides
Domain Adapta,on for Upper Body Pose Tracking in Signed TV

Domain Adapta,on for Upper Body Pose Tracking in Signed TV

Domain Adapta,on for Upper Body Pose Tracking in Signed TV Broadcasts James Charles 1 , Tomas Pfister 2 , Derek Magee 1 , David Hogg 1 , Andrew Zisserman 2 1 School

646 views • 50 slides
5a A&P: Introduction to the Human Body - Cells 5a A&P: Introduction to the Human Body -

5a A&P: Introduction to the Human Body - Cells 5a A&P: Introduction to the Human Body -

5a A&P: Introduction to the Human Body - Cells 5a A&P: Introduction to the Human Body - Cells Class Outline 5 minutes Attendance, Breath of Arrival, and Reminders 10 minutes Lecture: AOIs of the erectors 5

830 views • 70 slides
5a A&P: Introduction to the Human Body - Cells 5a A&P: Introduction to the Human Body -

5a A&P: Introduction to the Human Body - Cells 5a A&P: Introduction to the Human Body -

5a A&P: Introduction to the Human Body - Cells 5a A&P: Introduction to the Human Body - Cells Class Outline 5 minutes Attendance, Breath of Arrival, and Reminders 10 minutes Lecture: AOIs of the erectors 5

987 views • 71 slides
5a A&P Introduction to the Human Body 1 - Cells 5a A&P Introduction to the Human Body 1 -

5a A&P Introduction to the Human Body 1 - Cells 5a A&P Introduction to the Human Body 1 -

5a A&P Introduction to the Human Body 1 - Cells 5a A&P Introduction to the Human Body 1 - Cells Class Outline 5 minutes Attendance, Breath of Arrival, and Reminders 10 minutes Lecture: AOIs of the erectors 5

1.34k views • 71 slides
6a A&P: Introduction to the Human Body - Tissues 6a A&P: Introduction to the Human Body -

6a A&P: Introduction to the Human Body - Tissues 6a A&P: Introduction to the Human Body -

6a A&P: Introduction to the Human Body - Tissues 6a A&P: Introduction to the Human Body - Tissues Class Outline 5 minutes Attendance, Breath of Arrival, and Reminders 10 minutes Lecture: AOIs of the gluteals 5

831 views • 60 slides
The Light Spectrum Not all sensors are equal! 7 Human Body Temperature Human Body Temperature

The Light Spectrum Not all sensors are equal! 7 Human Body Temperature Human Body Temperature

The Light Spectrum Not all sensors are equal! 7 Human Body Temperature Human Body Temperature + Efficient Screenings + Non-Contact! + Effective - Not Magic - Precise Deployment - Not 100% FDA Recommendations Systems Work Effectively

558 views • 26 slides
Human Pose Estimation and Action Recognition Gang Yu, Megvii (Face++) Junsong Yuan, SUNY Buffalo

Human Pose Estimation and Action Recognition Gang Yu, Megvii (Face++) Junsong Yuan, SUNY Buffalo

ICIP 2019 Tutorial Human Pose Estimation and Action Recognition Gang Yu, Megvii (Face++) Junsong Yuan, SUNY Buffalo Zicheng Liu, Microsoft Overview Part1: Human Pose Part2: Action Recognitio n Estimation Datasets 2D Skeleton

998 views • 77 slides
Fields of Parts & Friends peter.gehler.net p i Detection + Geometry p i Human Pose

Fields of Parts & Friends peter.gehler.net p i Detection + Geometry p i Human Pose

Fields of Parts & Friends peter.gehler.net p i Detection + Geometry p i Human Pose Estimation or Predict Predict Observation Observation Bounding Boxes Joint Locations Human Pose Estimation F (1) top X Y top F (2) top , head . . .

564 views • 40 slides
Modeling Mutual Context of Object and Human Pose in Human-object Interaction Activities

Modeling Mutual Context of Object and Human Pose in Human-object Interaction Activities

Modeling Mutual Context of Object and Human Pose in Human-object Interaction Activities Bangpeng Yao Li Fei-Fei Presented by Sahil Shah Agenda Introduction Problem Formulation Learning Inference Results Agenda

536 views • 34 slides
Harmony (of the Self) with the Body - Understanding Prosperity Human Being Self (I) Body

Harmony (of the Self) with the Body - Understanding Prosperity Human Being Self (I) Body

Harmony (of the Self) with the Body - Understanding Prosperity Human Being Self (I) Body Co-existence Ekkuo uo eSaSa kjhj lgvfLrRo Need Happiness (e.g. Respect) Physical Facility (e.g. Food) vko';d ;drk lq lq[k tSls lEeku

862 views • 16 slides
The Gross Human Body, its relations with the Subtle Body common ailments & their remedies.

The Gross Human Body, its relations with the Subtle Body common ailments & their remedies.

The Gross Human Body, its relations with the Subtle Body common ailments & their remedies. Published for free online viewing & download at: www.scribd.com/presentation/335934148 The Human CNS exposed by dissection from the dorsal

489 views • 37 slides
12. Chez Le Docteur 12.1 The Human Body 12.2 French Adverbs 12.1 The Human Body Mme Legendre is

12. Chez Le Docteur 12.1 The Human Body 12.2 French Adverbs 12.1 The Human Body Mme Legendre is

12. Chez Le Docteur 12.1 The Human Body 12.2 French Adverbs 12.1 The Human Body Mme Legendre is not feeling well, so she goes to the doctor. Heres their conversation: Mme: Docteur, je ne me sens pas bien, jai mal la tte et

805 views • 19 slides
Object Representation Based On Gabor Wave Vector Binning : An Application to Human Head Pose

Object Representation Based On Gabor Wave Vector Binning : An Application to Human Head Pose

Object Representation Based On Gabor Wave Vector Binning : An Application to Human Head Pose Detection M. Dahmane and J. Meunier University of Montreal Introduction Head pose is important: Inferring important non verbal information

517 views • 23 slides

More recommend