deep convolutional poses for human
play

Deep Convolutional Poses for Human Interaction Recognition in - PowerPoint PPT Presentation

Oct 10, 2023 •447 likes •858 views

Deep Convolutional Poses for Human Interaction Recognition in Monocular Videos Marcel Sheeny de Moraes Supervisor: Neil Robertson HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Outline

  1. Deep Convolutional Poses for Human Interaction Recognition in Monocular Videos Marcel Sheeny de Moraes Supervisor: Neil Robertson

  2. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Outline • Introduction • Related Works • Methodology • Results • Conclusion and Future Works 2 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  3. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Outline • Introduction • Related Works • Methodology • Results • Conclusion and Future Works 3 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  4. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Introduction • Human Interaction Recognition • Surveillance • Human-Computer Interaction Automatic Video labeling • Kicking Hand shake High five 4 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  5. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Introduction • Goal of the project • To use Human Pose Estimation to recognize the Human Interaction in Monocular videos (RGB) . 5 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  6. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Outline • Introduction • Related Works • Methodology • Results • Conclusion and Future Works 6 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  7. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Related Works • Park and Arggawal, (Multimedia systems 2004) • Ellipse and Convex Hull features. • Hierarchical Bayesian Network. 9 types of interactions. • 78% of accuracy. • 7 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  8. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Related Works • Yun, et al. , (CVPR 2012) • Depth camera (kinect) to estimate the human pose. • 6 features from human pose. • Multiple Instance Learning. • 8 types of interactions. • 80% of accuracy using 3 frames. • 91% of accuracy whole sequence. 8 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  9. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Related Works • Hu, et al. , (ECCV 2014) • Yun, et al. (2012) dataset. • Positive actor features. • Hidden Markov Model to classify. • 76.2% of accuracy per frame and 83.3% for the whole sequence. • Zhu, et al., (AAAI 2016) • Yun, et al. (2012) dataset. • Deep LSTM network to recognize the interaction using the human pose estimation. • 90.41% of accuracy. 9 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  10. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Benchmark • Benchmark for the Two-Person Interaction dataset. Method Per frame Whole sequence 91.10% Yun, et al. (2012) 80.30% Hu, et al. (2014) 76.1% 83.33% 90.41% Zhu, et al. (2015) - 10 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  11. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Outline • Introduction • Related Works • Methodology • Results • Conclusion and Future Works 11 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  12. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Methodology 12 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  13. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Dataset • Two-person Interaction Detection Using Body-Pose Features (Yun, et al ., 2012). 8 interactions: approaching, departing, pushing, kicking, punching, • exchanging objects, hugging, and shaking hands. • 282 samples of interactions. Kicking Punching Hugging Shaking Hands 13 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  14. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Methodology 14 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  15. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Person Detection • “Faster R -CNN: Towards Real-Time Object Detection with Region Proposal Networks”, by Ren, et al (2015) • State-of-the-art method for object detection. • 0.1 s to detect the person in each image. • Very Deep Convolutional Neural Network (VGG-16). 15 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  16. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Person Detection Results 16 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  17. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Methodology 17 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  18. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Multi-Person Tracking • Kalman Filter with a linear motion model. • Hungarian Algorithm is used to assign detections and predictions. • Threshold methods are used to decide new/lost tracks. 18 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  19. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Methodology 19 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  20. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Human Pose Estimation • “Convolutional Pose Machines”, by Wei, et al (CVPR 2016) • 12 hierarchical Deep Convolutional Neural Networks (DCNN). • 12 different size inputs. • Current state-of-the-art for human pose estimation. PCKh @ 0.2 PC Benchmark using LSP dataset Method Head Shoulder Elbow Wrist Hip Knee Ankle Total AUC Pischulin, et al., ICCV’13 87.2 56.7 46.7 38.9 61.0 57.5 52.7 57.1 35.8 Chen and Yulle, NIPS’14 91.8 78.2 71.8 65.5 73.3 70.2 63.4 73.4 40.1 Carreira, et al. , CVPR’16 90.5 81.8 65.8 59.8 81.6 70.6 62.0 73.1 41.5 Fan et al. , CVPR’15 92.4 75.2 65.3 64.0 75.7 68.3 70.4 73.0 42.2 Tompson, et al., NIPS’14 90.6 79.2 67.9 63.4 69.5 71.0 64.2 72.3 47.3 Yang, et al. , CVPR’16 90.6 78.1 73.8 68.8 74.8 69.9 58.9 73.6 39.3 Pischulin, et al. , CVPR’16 97.0 91.0 83.8 78.1 91.0 86.7 82.0 87.1 63.5 Wei, et al. , CVPR 97.8 92.5 87.0 83.9 91.5 90.8 89.9 90.5 65.4 20 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  21. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Human Pose Estimation • “Convolutional Pose Machines”, by Wei, et al (CVPR 2016) 21 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  22. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Results 22 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  23. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Results https://www.youtube.com/watch?v=llLj50gE9GI 23 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  24. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Methodology 24 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  25. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Feature Extraction • 6 types of features were used • XY Joint Position (XY) • Distances from Related Joints (DRJ) • Distances from One Joint (DOJ) • Absolute Difference (AD) • Joint Angles (JA) • Velocity (VEL) 25 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  26. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Joint Position and Distance from Related Joints • Raw Joint Position (XY) 𝐺 𝑘 = 𝑄(𝑘) • Distance Related Joints (DRJ) 1 𝑘 − 𝑄 2 𝑘 | 𝐺 𝑘 = | 𝑄 26 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  27. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Distance from One Joint and Absolute Difference • Distance from One Joint (DOJ) 1 𝑘 1 − 𝑄 2 𝑘 2 | 𝐺 𝑘 1 , 𝑘 2 = | 𝑄 • Absolute difference (AD) 1 𝑘 − 𝑄 2 𝑘 | 𝐺 𝑘 = |𝑄 27 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  28. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Joint Angles and Velocity • Joint Angles (JA) 𝑄 𝑧 𝑘 1 −𝑄 𝑧 (𝑘 2 ) 𝐺 𝑘 1 , 𝑘 2 = 𝑢𝑏𝑜 −1 𝑄 𝑦 𝑘 1 −𝑄 𝑦 (𝑘 2 ) • Velocity (VEL) 𝐺(𝑘, 𝑢 1 , 𝑢 2 ) = 𝑄 𝑘, 𝑢 1 − 𝑄(𝑘, 𝑢 2 ) 28 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

  29. HERIOT-WATT Introduction Related Works Methodology UNIVERSITY Results Conclusion and Future Works Methodology 29 /39 VIVA 15th June 2016 - Marcel Sheeny de Moraes

Recommend

CS7015 (Deep Learning) : Lecture 13 Visualizing Convolutional Neural Networks, Guided

CS7015 (Deep Learning) : Lecture 13 Visualizing Convolutional Neural Networks, Guided

CS7015 (Deep Learning) : Lecture 13 Visualizing Convolutional Neural Networks, Guided Backpropagation, Deep Dream, Deep Art, Fooling Convolutional Neural Networks Mitesh M. Khapra Department of Computer Science and Engineering Indian Institute

1.07k views • 72 slides
of Objects and Human Poses Maryam Daneshi, Konstantin Bayandin May 28 th , 2013 1 Agenda

of Objects and Human Poses Maryam Daneshi, Konstantin Bayandin May 28 th , 2013 1 Agenda

Recognizing Human-Object Interactions in Still Images by Modeling the Mutual Context of Objects and Human Poses Maryam Daneshi, Konstantin Bayandin May 28 th , 2013 1 Agenda Introduction & Motivation Dataset description Model

935 views • 36 slides
Lifting from the Deep: Convolutional 3D Pose Estimation from a Single Image Denis Tom

Lifting from the Deep: Convolutional 3D Pose Estimation from a Single Image Denis Tom

Lifting from the Deep: Convolutional 3D Pose Estimation from a Single Image Denis Tom Chris Russell Lourdes Agapito We introduce a novel approach to solve the problem of 3D human pose estimation from a single RGB image Input

841 views • 18 slides
11/21/2018 ImageNet Classification with Deep Convolutional Neural Networks Prepared by Faizaan

11/21/2018 ImageNet Classification with Deep Convolutional Neural Networks Prepared by Faizaan

11/21/2018 ImageNet Classification with Deep Convolutional Neural Networks Prepared by Faizaan Naveed Won Mo (Andy) Jung Lassonde School of Engineering, York University Canada Objective Overview of Convolutional Neural Network

567 views • 17 slides
Convolutional Networks Lecture slides for Chapter 9 of Deep Learning Ian Goodfellow 2016-09-12

Convolutional Networks Lecture slides for Chapter 9 of Deep Learning Ian Goodfellow 2016-09-12

Convolutional Networks Lecture slides for Chapter 9 of Deep Learning Ian Goodfellow 2016-09-12 Convolutional Networks Scale up neural networks to process very large images / video sequences Sparse connections Parameter sharing

552 views • 27 slides
Food Image Recognition Using Very Deep Convolutional Networks Hamid Hassannejad 2 nd

Food Image Recognition Using Very Deep Convolutional Networks Hamid Hassannejad 2 nd

Food Image Recognition Using Very Deep Convolutional Networks Hamid Hassannejad 2 nd International Workshop on Multimedia Assisted Dietary Management Oct. 2016 Outlines Deep learning and food recognition Inception model Experiments

704 views • 16 slides
Multiscale Sparse Models in Deep Convolutional Networks Tomas Angls, Roberto Leonarduzi,

Multiscale Sparse Models in Deep Convolutional Networks Tomas Angls, Roberto Leonarduzi,

Multiscale Sparse Models in Deep Convolutional Networks Tomas Angls, Roberto Leonarduzi, Stphane Mallat, Louis Thiry, John Zarka, Sixin Zhang Collge de France cole Normale Suprieure Flatiron Institute www.di.ens.fr/data Deep

356 views • 33 slides
Deep Convolutional Networks are Useful in System Identification Antnio H. Ribeiro 1 , 2 , ,

Deep Convolutional Networks are Useful in System Identification Antnio H. Ribeiro 1 , 2 , ,

Deep Convolutional Networks are Useful in System Identification Antnio H. Ribeiro 1 , 2 , , Carl Andersson 1 , , Koen Tiels 1 , Niklas Wahlstrm 1 and Thomas B. Schn 1 1 Uppsala University, 2 UFMG, Equal contribution

332 views • 29 slides
Understanding (or not) Deep Convolutional Networks Stphane Mallat cole Normale Suprieure

Understanding (or not) Deep Convolutional Networks Stphane Mallat cole Normale Suprieure

Understanding (or not) Deep Convolutional Networks Stphane Mallat cole Normale Suprieure www.di.ens.fr/data Deep Neural Networks Approximations of high-dimensional functions from examples, for classification and regression.

387 views • 38 slides
High-Resolution Breast Cancer Screening with Multi-View Deep Convolutional Neural Networks

High-Resolution Breast Cancer Screening with Multi-View Deep Convolutional Neural Networks

High-Resolution Breast Cancer Screening with Multi-View Deep Convolutional Neural Networks Krzysztof J. Geras Joint work with Kyunghyun Cho , Linda Moy , Gene Kim , Stacey Wolfson and Artie Shen . GTC 2017 W HERE DEEP LEARNING IS USEFUL amount

772 views • 42 slides
Lecture 13: Introduction to Deep Learning Deep Convolutional Neural Networks Aykut Erdem

Lecture 13: Introduction to Deep Learning Deep Convolutional Neural Networks Aykut Erdem

Lecture 13: Introduction to Deep Learning Deep Convolutional Neural Networks Aykut Erdem November 2016 Hacettepe University Administrative Assignment 3 is out! It is due November 30, 2016 You will implement a 2-layer Neural

1.28k views • 114 slides
Fully Convolutional Network (FCN) Prof. Seungchul Lee Industrial AI Lab. Deep Learning for

Fully Convolutional Network (FCN) Prof. Seungchul Lee Industrial AI Lab. Deep Learning for

Fully Convolutional Network (FCN) Prof. Seungchul Lee Industrial AI Lab. Deep Learning for Computer Vision: Review Source: 6.S191 Intro. to Deep Learning at MIT 2 Segmentation Segmentation task is different from classification task because

522 views • 15 slides
I mproving*Object*Detection*with* Deep*Convolutional*Networks*via* Bayesian*Optimization*and*

I mproving*Object*Detection*with* Deep*Convolutional*Networks*via* Bayesian*Optimization*and*

I mproving*Object*Detection*with* Deep*Convolutional*Networks*via* Bayesian*Optimization*and* Structured*Prediction* Yuting Zhang * ,*KihyukSohn ,*Ruben*Villegas ,*Gang*Pan * ,* HonglakLee *

860 views • 84 slides
Deep Convolutional Neural Network for Computer Vision Products LI XU, R&D Director

Deep Convolutional Neural Network for Computer Vision Products LI XU, R&D Director

Deep Convolutional Neural Network for Computer Vision Products LI XU, R&D Director SenseTime Group Limited SenseTime Introduction SenseTime focuses on invention and development of computer vision and deep learning technologies. Our

1.03k views • 39 slides
Underwater sparse image classification using deep convolutional neural networks Mohamed Elawady

Underwater sparse image classification using deep convolutional neural networks Mohamed Elawady

Underwater sparse image classification using deep convolutional neural networks Mohamed Elawady Heriot-Watt University VIBOT Msc 2014 26 Nov 2015 Deep Learning Workshop, Lyon, 2015 1 About Me! 2014-Current: PhD in Imaging Processing

740 views • 47 slides
Convolutional Neural Networks (Part III) 08, 10 & 17 Nov, 2016 J. Ezequiel Soto S. Image

Convolutional Neural Networks (Part III) 08, 10 & 17 Nov, 2016 J. Ezequiel Soto S. Image

Convolutional Neural Networks (Part III) 08, 10 & 17 Nov, 2016 J. Ezequiel Soto S. Image Processing 2016 Prof. Luiz Velho Convolutional Neural Networks 1 Summary & References 08/11 ImageNet Classification with Deep Convolutional

444 views • 27 slides
Deep Learning for Predictive Maintenance Pawel Morkisz GTC 2017 Agenda Problem

Deep Learning for Predictive Maintenance Pawel Morkisz GTC 2017 Agenda Problem

Deep Learning for Predictive Maintenance Pawel Morkisz GTC 2017 Agenda Problem Introduction and notion of deep neural networks o Convolutional layers o Residual networks (ResNet) One dimensional convolutional networks in failure

276 views • 23 slides
SPP-net Spatial Pyramid Pooling in Deep Convolutional Networks Kaiming He Xiangyu Zhang

SPP-net Spatial Pyramid Pooling in Deep Convolutional Networks Kaiming He Xiangyu Zhang

SPP-net Spatial Pyramid Pooling in Deep Convolutional Networks Kaiming He Xiangyu Zhang Shaoqing Ren Jian Sun Microsoft Research Asia Visual Computing Group Highlights ILSVRC 2014 (all provided-data tracks) DET - 2 nd CLS - 3 rd

488 views • 13 slides
SnaPEA : Predictive Early Activation for Reducing Computation In Deep Convolutional Neural

SnaPEA : Predictive Early Activation for Reducing Computation In Deep Convolutional Neural

SnaPEA : Predictive Early Activation for Reducing Computation In Deep Convolutional Neural Networks * * Vahideh Akhlaghi Amir Yazdanbakhsh Kambiz Samadi Rajesh K. Gupta Hadi Esmaeilzadeh * Equal Contribution University

565 views • 32 slides
Evaluations of Deep Convolutional Neural Networks for Automatic Identification of Malaria Infected

Evaluations of Deep Convolutional Neural Networks for Automatic Identification of Malaria Infected

Evaluations of Deep Convolutional Neural Networks for Automatic Identification of Malaria Infected Cells Yuhang Dong, Zhuocheng Jiang, Hongda Shen, W. David Pan Dept. of Electrical & Computer Engineering, Univ. of Alabama in Huntsville (UAH)

370 views • 14 slides
Return of the Devil in the Details: Delving Deep into Convolutional Nets Ken Chatfield, Karen

Return of the Devil in the Details: Delving Deep into Convolutional Nets Ken Chatfield, Karen

Return of the Devil in the Details: Delving Deep into Convolutional Nets Ken Chatfield, Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman Visual Geometry Group, Department of Engineering Science, Univesity of Oxford Hilal E. Akyz 1 2

908 views • 39 slides
An Ensemble of Deep Convolutional Networks for Automatic Film Score Genre Recognition Toward a

An Ensemble of Deep Convolutional Networks for Automatic Film Score Genre Recognition Toward a

An Ensemble of Deep Convolutional Networks for Automatic Film Score Genre Recognition Toward a Discovery-Rich Recommendation Agent Problem: Background Track Bottleneck Video and audio are growing exponentially Overwhelming choice leads to

398 views • 38 slides
Classification Accuracies of Malaria Infected Cells Using Deep Convolutional Neural Networks

Classification Accuracies of Malaria Infected Cells Using Deep Convolutional Neural Networks

Classification Accuracies of Malaria Infected Cells Using Deep Convolutional Neural Networks Based on Decompressed Images Yuhang Dong, Zhuocheng Jiang, Hongda Shen, W. David Pan Dept. of Electrical & Computer Engineering, Univ. of Alabama

660 views • 18 slides
RECOGNIZING PORNOGRAPHIC IMAGES USING DEEP CONVOLUTIONAL NEURAL NETWORKS Olarik Surinta and

RECOGNIZING PORNOGRAPHIC IMAGES USING DEEP CONVOLUTIONAL NEURAL NETWORKS Olarik Surinta and

RECOGNIZING PORNOGRAPHIC IMAGES USING DEEP CONVOLUTIONAL NEURAL NETWORKS Olarik Surinta and Thananchai Khamket Faculty of Informatics Mahasarakham University th Present at the 4 International Conference on Digital Arts, Media and

618 views • 22 slides

More recommend