Dance Movement Patterns Recognition (Part II) Jess Snchez Morales - PowerPoint PPT Presentation

Dance Movement Patterns Recognition (Part II) Jesús Sánchez Morales EEMCS - HMI

Contents Contents • Goals Goals • • HMM HMM • • Recognizing Simple Steps Recognizing Simple Steps • • Recognizing Complex Patterns Recognizing Complex Patterns • • Auto Generation of Complex Patterns Graphs Auto Generation of Complex Patterns Graphs • • Test Bench Test Bench • • Conclusions Conclusions • 2/32 EEMCS - HMI

Contents Contents • Goals Goals • • HMM HMM • • Recognizing Simple Steps Recognizing Simple Steps • • Recognizing Complex Patterns Recognizing Complex Patterns • • Auto Generation of Complex Patterns Graphs Auto Generation of Complex Patterns Graphs • • Test Bench Test Bench • • Conclusions Conclusions • 3/32 EEMCS - HMI

Goals Goals • Recognizing simple user Recognizing simple user’ ’s movements s movements • • Recognizing complex patterns Recognizing complex patterns • • Auto generation of reference patterns Auto generation of reference patterns • • Pattern searching during the dance without Pattern searching during the dance without • any reference any reference 4/32 EEMCS - HMI

Contents Contents • Goals Goals • • HMM HMM • • Recognizing Simple Steps Recognizing Simple Steps • • Recognizing Complex Patterns Recognizing Complex Patterns • • Auto Generation of Complex Patterns Graphs Auto Generation of Complex Patterns Graphs • • Test Bench Test Bench • • Conclusions Conclusions • 5/32 EEMCS - HMI

Hidden Markov Model (HMM) Hidden Markov Model (HMM) • Highly used in Speech Recognition • Highly used in Speech Recognition • We adapted it for our problem • We adapted it for our problem • Data sequences to analyze • Data sequences to analyze Stop move 1 move 2 stop • HMM graphs for each recognition • HMM graphs for each recognition x2% x3% x5% • Viterbi’ ’s s algorithm algorithm • Viterbi (1-x2)% (1-x3)% x1% Initial State State Final State 1 2 State (1-x1)% (1-x4)% (1-x5)% State 3 x4% 6/32 EEMCS - HMI

Contents Contents • Goals Goals • • HMM HMM • • Recognizing Simple Steps Recognizing Simple Steps • • Recognizing Complex Patterns Recognizing Complex Patterns • • Auto Generation of Complex Patterns Graphs Auto Generation of Complex Patterns Graphs • • Test Bench Test Bench • • Conclusions Conclusions • 7/32 EEMCS - HMI

Recognizing Simple Steps Recognizing Simple Steps • What we understand as being a simple What we understand as being a simple • step? step? – Right, left, down, up, jump and twister Right, left, down, up, jump and twister – • How we divided these recognitions How we divided these recognitions • – Horizontal Horizontal – – Vertical Vertical – – Twister Twister – 8/32 EEMCS - HMI

Recognizing simple steps (Horizontal) Recognizing simple steps (Horizontal) • Recognized steps Recognized steps • – Left step Left step – – Right step Right step – • Used data Used data • – Horizonal Horizonal variation of the centre of mass between variation of the centre of mass between – frames frames – User radius User radius – 9/32 EEMCS - HMI

Recognizing simple steps (Horizontal) Recognizing simple steps (Horizontal) • Building a sequence and launching a Building a sequence and launching a • thread thread • Markov graph Markov graph • 10/32 EEMCS - HMI

Recognizing simple steps (Vertical) Recognizing simple steps (Vertical) • Recognized steps Recognized steps • – Up Up – – Down Down – – Jump Jump – • Used data Used data • – Vertical position of the centre of mass Vertical position of the centre of mass – – Vertical average position of the centre of mass Vertical average position of the centre of mass – – User radius User radius – 11/32 EEMCS - HMI

Recognizing simple steps (Vertical) Recognizing simple steps (Vertical) • Building a sequence and launching a Building a sequence and launching a • thread thread • Markov graph Markov graph • 12/32 EEMCS - HMI

Recognizing simple steps (Twister) Recognizing simple steps (Twister) • Recognized steps • Recognized steps – Twister Twister – • Used data • Used data – User radius – User radius – Average radius – Average radius – Horizontal and vertical position of the centre of mass – Horizontal and vertical position of the centre of mass . 13/32 EEMCS - HMI

Recognizing simple steps (Twister) Recognizing simple steps (Twister) • Building a sequence and launching a Building a sequence and launching a thread thread • Markov graph Markov graph • 14/32 EEMCS - HMI

Contents Contents • Goals Goals • • HMM HMM • • Recognizing Simple Steps Recognizing Simple Steps • • Recognizing Complex Patterns Recognizing Complex Patterns • • Auto Generation of Complex Patterns Graphs Auto Generation of Complex Patterns Graphs • • Test Bench Test Bench • • Conclusions Conclusions • 15/32 EEMCS - HMI

Recognizing Complex Patterns Recognizing Complex Patterns • What we understand as being complex What we understand as being complex • pattern? pattern? – Combinations of simple steps Combinations of simple steps – Left Step + Right Step + Left Step + Jump + Twister +… … Left Step + Right Step + Left Step + Jump + Twister + • What we receive from the simple step What we receive from the simple step • recognition recognition Left Step – Step code Step code – Code: 11 – Duration Duration – – Step time Step time – Duration: 3 frames Time: second 54 16/32 EEMCS - HMI

Recognizing Complex Patterns Recognizing Complex Patterns • Building a sequence and launching a thread Building a sequence and launching a thread • • Markov Graph Markov Graph • . 17/32 EEMCS - HMI

Contents Contents • Goals Goals • • HMM HMM • • To Recognize Simple Steps To Recognize Simple Steps • • To Recognize Complex Patterns To Recognize Complex Patterns • • Auto Generation of Complex Patterns Graphs Auto Generation of Complex Patterns Graphs • • Test Bench Test Bench • • Conclusions Conclusions • 18/32 EEMCS - HMI

Auto Generation of Complex Patterns Auto Generation of Complex Patterns Graphs Graphs • Easy way of building graphs for Easy way of building graphs for • complex patterns recognition complex patterns recognition • Included in the pattern recognition Included in the pattern recognition • • Saved as a text file Saved as a text file • 19/32 EEMCS - HMI

Auto Generation of Complex Patterns Auto Generation of Complex Patterns Graphs Graphs • We receive the sequence of simple We receive the sequence of simple • steps steps • We build the graph for the recognition We build the graph for the recognition • 20/32 EEMCS - HMI

Contents Contents • Goals Goals • • HMM HMM • • Recognizing Simple Steps Recognizing Simple Steps • • Recognizing Complex Patterns Recognizing Complex Patterns • • Auto Generation of Complex Patterns Graphs Auto Generation of Complex Patterns Graphs • • Test Bench Test Bench • • Conclusions Conclusions • 21/32 EEMCS - HMI

Test Bench Test Bench • Test setup Test setup • – Good external conditions Good external conditions – – Real time tests Real time tests - - 210 steps 210 steps – – Possible results for each test Possible results for each test – • Well detected Well detected • • Wrong detected Wrong detected • • Not detected Not detected • 22/32 EEMCS - HMI

Test Bench Test Bench • Test organization Test organization • – Horizontal recognition Horizontal recognition – – Vertical recognition Vertical recognition – – Twister recognition Twister recognition – – Complex pattern recognition Complex pattern recognition – 23/32 EEMCS - HMI

Test Bench (Horizontal recognition) Test Bench (Horizontal recognition) • High success rate High success rate • • No wrong detections No wrong detections • • In case of fast dance some steps get In case of fast dance some steps get • lost lost Individual steps Various steps (slow) Various steps (fast) 0,00% 0,00% 0,00% Well detected 19,23% 0,00% 0,00% No detected Wrong detected 80,77% 100,00% 100,00% 24/32 EEMCS - HMI

Test Bench (Vertical recognition) Test Bench (Vertical recognition) • Very good individual step recognition Very good individual step recognition • • Slow dance tests: some recognition problems Slow dance tests: some recognition problems • • Fast dance tests: recognition performance Fast dance tests: recognition performance • decreases decreases Individual steps Various steps (slow) Various steps (fast) Well detected 12,50% 15,38% 18,18% 6,25% No detected 0,00% Wrong detected 53,85% 30,77% 81,82% 81,25% 25/32 EEMCS - HMI