Motion capture: An evaluation of Kinect V2 body tracking for upper - PowerPoint PPT Presentation

Motion capture: An evaluation of Kinect V2 body tracking for upper limb motion analysis Silvio Giancola 1 , Andrea Corti 1 , Franco Molteni 2 , Remo Sala 1 1 Vision Bricks Laboratory, Mechanical Departement, Politecnico di Milano 2 Movement Analysis Lab of Valduce Hospital "Villa Beretta" Rehabilitation Centre An evaluation of Kinect V2 body tracking for upper limb motion analysis

2 / 25 Summary ry • Introduction on Movement Analysis and 3D Computer Vision • Vision systems for Motion Capture • Multi-View Stereoscopic system : BTS Smart-DX 7000 • Time-of-Flight Camera: Microsoft Kinect V2 • Kinematics of the upper limb • Experimental Setup • Wrist position measurement • Elbow angle measurement • Uncertainty Estimation • Wrist position • Elbow angle • Conclusion • References An evaluation of Kinect V2 body tracking for upper limb motion analysis

3 / 25 Summary In Introduction on Movement Analysis • Introduction • Study of the capacity of a person to realize a determined movement. • Vision systems for Motion Capture • It analyses of the kinematics and/or dynamics of the human body. • BTS Smart-DX 7000 • Microsoft Kinect V2 • Kinematics analysis • Kinematics analysis  Range of motion • Experimental Setup  Absolute position in space • Wrist position  Speed, Acceleration, Jerk measurement • Elbow angle measurement • Dynamics related to the movement, • Uncertainty Estimation using inertia and mass information • Wrist position and external forces measurement • Elbow angle  Forces/Moments on the articulation  Forces applied on the muscles • Conclusion • References An evaluation of Kinect V2 body tracking for upper limb motion analysis

4 / 25 Summary In Introduction on Movement Analysis • Introduction • Valduce Hospital “ Villa Beretta ” • Vision systems for • Rehabilitation Centre Motion Capture • BTS Smart-DX 7000 • Costa Masnaga (LC) • Microsoft Kinect V2 • Gait and Movement Analysis Laboratory • Kinematics analysis • Dynamic EMG / 3D Motion analysis • Evaluates causes of walking and related movement problems • Experimental Setup • Analyses muscle function and dexterity • Wrist position measurement • Focuses on patients with nervous system damage, • Elbow angle • Provides Basis for corrective physical, medical and surgical therapies measurement • Uncertainty Estimation • Wrist position • Elbow angle • Conclusion • References An evaluation of Kinect V2 body tracking for upper limb motion analysis

5 / 25 Summary In Introduction on 3D Computer Vision • Introduction Vision systems are non contact optical measurement techniques • Vision systems for + With no loading effect on the measured system (unlike IMUs) Motion Capture • BTS Smart-DX 7000 + With no damage on the measured system • Microsoft Kinect V2 + Remotely measures simultaneous points – Sensible to occlusion, material reflection and light conditions • Kinematics analysis • Experimental Setup 3D Vision systems: • Wrist position measurement • Permits the measurement of the position of points in a scene • Elbow angle • Deals with point clouds : a set of 3D points, that can be: measurement • Dense and structured • Uncertainty Estimation → Matrix of points • Sparse and unstructured • Wrist position → Small array of points • Elbow angle • Allows body recognition • Conclusion and position estimation in 3D • References An evaluation of Kinect V2 body tracking for upper limb motion analysis

6 / 25 Summary In Introduction on 3D Computer Vision • Introduction Different techniques exist: • Vision systems for • Multi-View Stereoscopy: Motion Capture • BTS Smart-DX 7000 • 2 or more cameras • Microsoft Kinect V2 • Sparse point cloud reconstruction • Kinematics analysis • Active triangulation: • Experimental Setup • Single camera with structured light projector • Wrist position • Laser blade for static scene measurement • Codified light pattern for dynamic scene • Elbow angle measurement • Uncertainty Estimation • Time of Flight camera (TOF): • Wrist position • Light echo measurement (LiDAR) • Time between emission of modulated light and its reception • Elbow angle • Diffused light • Conclusion • Dense point cloud reconstruction • References An evaluation of Kinect V2 body tracking for upper limb motion analysis

7 / 25 Summary Vision systems for Motion Capture • Introduction • Vision systems for Stereoscopic system Motion Capture • BTS Smart-DX 7000 • BTS Smart-DX 7000 • Up to 16 cameras • Microsoft Kinect V2 • Resolution of 2048 x 2048 pixels • Kinematics analysis • Up to 2000 fps (500 fps at full frame) • Experimental Setup • Precision under 0.1 mm • Wrist position • Volume of 6 x 6 x 3 m measurement • Strobe wavelength of 850 nm (IR light) • Elbow angle measurement • Uncertainty Estimation • Set of reflective markers fixed on the body • Wrist position • Elbow angle • Conclusion • References An evaluation of Kinect V2 body tracking for upper limb motion analysis

8 / 25 Summary Vision systems for Motion Capture • Introduction Stereoscopic system • Vision systems for • It exists plenty of Marker Placement Motion Capture • BTS Smart-DX 7000 • Body Segment CM • Microsoft Kinect V2 • Plug-in-Gait • Helen Hayes (Davis) • Kinematics analysis • Cleveland Clinic Model • Experimental Setup • Golfer Full-Body • Wrist position • …. measurement • Elbow angle measurement • Main drawbacks: • Uncertainty Estimation • Measurements need to be realized on the skeleton , not on marker fixed • Wrist position on soft tissues that are not even rigid respect to the skeleton. • Markers are numerous, complicated and fastidious to fix, and results • Elbow angle depend entirely on how these marker are fixed. • Conclusion • Expensive (>200 k€) • References An evaluation of Kinect V2 body tracking for upper limb motion analysis

9 / 25 Summary Vision systems for Motion Capture • Introduction • Vision systems for TOF Systems Motion Capture • Microsoft Kinect V2 • BTS Smart-DX 7000 • RGB-D camera • Microsoft Kinect V2 • 512 x 424 pixels, up to 30 Hz • Kinematics analysis • Range from 0.5 to 4.5m - Field of view 70 ° x 60 ° • Experimental Setup • Precision around 2 mm for the point cloud reconstruction • Wrist position • A. Corti, S. Giancola, G. Mainetti, R. Sala, “ A metrological characterization of the Kinect V2 time-of- measurement flight camera”, Robotics and Autonomous Systems , vol. 75, pp. 584-594, 2016. • Elbow angle measurement • Integrated software for markerless human-motion capture • Uncertainty Estimation • Human body seen with 25 joints • Wrist position • On-line elaboration for real-time application • Elbow angle • Machine learning black box feed with thousands of bodies (adults) • Conclusion • J. Shotton, T. Sharp, A. Kipman, A. Fitzgibbon, M. Finocchio, A. Blake, M. Cook, R. Moore, “Real -time human pose recognition in parts from single depth images“ , Communications of the ACM , vol. 56, • References no. 1, pp. 116-124, 2013. An evaluation of Kinect V2 body tracking for upper limb motion analysis

10 / 25 Summary Vision systems for Motion Capture • Introduction • Vision systems for TOF Systems Motion Capture • Markerless human-motion tracking • BTS Smart-DX 7000 • Fully tracks up to six bodies simultaneously • Microsoft Kinect V2 • Tracks up to 25 joints • Kinematics analysis • Position in 3D space (in meters) • Experimental Setup • Absolute orientation (in quaternion) • Wrist position • (Hand state tracking) measurement • (Face recognition) • Elbow angle • Low cost (200 €) measurement • Uncertainty Estimation • Wrist position • Elbow angle • Conclusion • References An evaluation of Kinect V2 body tracking for upper limb motion analysis

11 / 25 Summary Villa Beretta - Experimental Setup • Introduction • Vision systems for Gait and Movement Analysis Laboratory Motion Capture 1. BTS Smart-DX 7000 (8 cameras – 250 fps) • BTS Smart-DX 7000 2. Footboard platform • Microsoft Kinect V2 3. Vision system for video recording • Kinematics analysis • Experimental Setup • Wrist position 1 1 measurement • Elbow angle 4 Our Setup measurement 4. Kinect V2 • Uncertainty Estimation Single 3D camera 3 • Wrist position 30 fps 3 • Elbow angle • Conclusion • References 2 An evaluation of Kinect V2 body tracking for upper limb motion analysis