Kitchen Scene Context based Gesture Recognition ICPR2012 Contest 1 - PowerPoint PPT Presentation

Kitchen Scene Context based Gesture Recognition ICPR2012 Contest 1

Overview of the contest 2

Organizers • Atsushi Shimada – Kyushu University, Japan • Kazuaki Kondo – Kyoto University, Japan • Daisuke Deguchi – Nagoya University, Japan • Géraldine Morin – IRIT-ENSEEIHT , France • Helman Stern – Ben Gurion University of the Negev, Israel 3

Special Thanks • Actors – Tomo Asakura (Kyoto Univ.) – Keizaburo Takase (Kyoto Univ.) – Keisuke Yasuzawa (Kyoto Univ.) – Kenta Matsui (Kyoto Univ.) – Yusa Ko (Kyoto Univ.) • Illustrations – Toshiki Sonoda (Kyushu Univ., KUCC) 4

Contest Background • Can gesture recognition be achieved by motion analysis alone? • Importance of scene context for gesture recognition – Strong relationship between action and situation surrounding an actor 5

Subject • Gesture recognition in Kitchen – Motion analysis – Recognition of ingredients, cooking tools, etc. 6

Cooking Menus • ham and eggs • omelet • scrambled egg • boiled egg • Kinshi-tamago 7

Contest Task • Label assignment to video frames 2 - mixing 3 - baking 4 - turning 1 - breaking 5 - cutting 6 - boiling 7 - seasoning 8 - pealing 8

Dataset • RGBD video sequence – Captured by Kinect @ 30fps • Each cooking – 5-10 minute long (2,000 to 12,000 frames) 9

Dataset • Five menus • Training set – Five actors – Ground truth was given • Test set – Two actors – Ground truth was NOT given 10

Evaluation • F-measure – Harmonic mean of Precision & Recall • For cooking menu � � ��� 11

Applications • 6 teams – India – Singapore – Japan x 3 – Taiwan • Invitation – 4 teams 12

Evaluation Result 1 0.8 0.6 0.4 0.2 0 Motion feature only 13

Evaluation Result in Each Gesture 1 0.8 ID ‐ 01 0.6 ID ‐ 02 0.4 ID ‐ 03 0.2 ID ‐ 04 ID ‐ 05 0 Reference 14

Awards ceremony 15

Awards • Sponsored by SIG-CEA of HCG, IEICE, Japan – SIG-CEA A study group of “Cooking and Eating Activities” • Researching “CEA” from viewpoint of • engineering, social and domestic science Supporting/improving dietary life with • information media technology – Activity Domestic meeting: 4 times a year • International Workshop: once a year • CEA2009, 2010, 2011, 2012 • 16

First Prize • Keisuke DOMAN, Chengying KUAI – Chukyo University, Brother Industries, Ltd. – “Kitchen Scene Context based Gesture Recognition” Waterproof TV 17

Second Prize • Tam V. Nguyen, Zheng Song, Manthan Pancholi, Li Zhang, Shuicheng Yan – National University of Singapore – “Context-aware Action Recognition in Kitchen Environment” Thermal mug Steam Roaster 18

Honorable Mention • Shubham Bansal, Shubham Gupta, Shubham Khandelwal, Dushyant Goyal – The LNM Institute of Information Technology – “Kitchen Activity Recognition based on Scene Context” Microwave Egg Boiler 19

Honorable Mention • Soichiro Hotta – Mie University – GLAC feature with SVM for Kitchen Gesture Recognition Microwave Egg Boiler 20

Results of Winners (Total) 1 0.8 0.6 0.4 0.2 0 Doman et al. Nguyen et al. Bansal et al. Hotta et al. 21

Results of Winners (Each Gesture) 1 0.8 0.6 Doman et al. 0.4 Nguyen et al. 0.2 Bansal et al. Hotta et al. 0 22