Capturing full body motion Antoine Kaufmann antoinek@student.ethz.ch April 9, 2013 Distributed Systems Seminar 1
What is Motion Capture? April 9, 2013 Distributed Systems Seminar 2
What is Motion Capture? Motion capture is the process of recording the movement of objects or people... In filmmaking and video game development, it refers to recording actions of human actors, Sources: http://lukemccann.wordpress.com/motion-capture/ and using that information to animate digital character models in 2D or 3D computer animation. [Wikipedia: Motion Capture] April 9, 2013 Distributed Systems Seminar 3
What is it used for? April 9, 2013 Distributed Systems Seminar 4
April 9, 2013 Applications: Filmmaking Distributed Systems Seminar 5 Sources: http://www.fxguide.com/featured/the-hobbit-weta/ http://www.ugo.com/therush/avatar-moments-that-give-us-pause-6-gallery http://www.animationmagazine.net/events/ted-ruffles-feathers-at-the-oscars/
April 9, 2013 Applications: Game development Distributed Systems Seminar 6 Sources: http://www.shipwrckd.com/2012/09/capturedinto-ubisofts-motion-capture-studio-launch- party/ http://gamerant.com/bioshock-infinite-elizabeth-trailer/
April 9, 2013 Applications: Gaming Distributed Systems Seminar 7 Sources: http://123kinect.com/kinect-sports-season-achivements/25170/ http://pikigeek.com/2012/03/06/peter-molyneux-says-we-need-more-kinect-games-or-we-will-die/
April 9, 2013 Applications: Virtual Reality Distributed Systems Seminar 8 Sources: http://www.wired.com/dangerroom/2012/01/army-virtual-reality/ http://www.doolwind.com/blog/where-is-virtual-reality/
April 9, 2013 Applications: Biomechanics Distributed Systems Seminar 9 Sources: http://orthopedics.childrenscolorado.org/our-programs/center-for-gait-and-movement- analysis http://www.motionanalysis.com/html/movement/movement.html
How can it be done? April 9, 2013 Distributed Systems Seminar 10
System Types: Mechanical Sources: http://www.metamotion.com/gypsy/gypsy-motion-capture-system-workflow.htm Potentiometers and exoskeleton � Accurate � Post-processing straight forward � No global root-motion � Restricts range of motion � April 9, 2013 Distributed Systems Seminar 11
System Types: Inertial Sources: Source:http://www.xsens.com/en/general/mvn � Multiple inertial measurement units (IMUs) to track motion � Gyroscopes or accelerometers Only relative position � � Problems with drift April 9, 2013 Distributed Systems Seminar 12
Problem: Drift Occurs if only relative measurements are available � Due to inaccuracies of sensors and calculations � Sources: Practical Motion Capture in Everyday Surroundings, Vlasic et al. 2007 April 9, 2013 Distributed Systems Seminar 13
System Types: Optical Sources: http://beforevfx.tumblr.com/image/44047276135 Usually cameras and markers � Markers are � passive (reflective) or � active (controlled LEDs) � Can achieve high level of detail � April 9, 2013 Distributed Systems Seminar 14
April 9, 2013 � System Types: Optical Problems: � � � Requires good contrast and lighting Marker-swapping Occlusion Distributed Systems Seminar 15 Sources: http://www.cgadvertising.com/pages/posts/vicon-technologies-give-usc-students-hands- on-motion-capture-experience128.php
System Types Mechanical � Inertial � Optical � Image based � Magnetic � � Acoustic Radio / Electromagnetic � Motion Tracking: No SIlver Bullet but a Respectable Arsenal by Greg Welch and Eric Foxlin April 9, 2013 Distributed Systems Seminar 16
What’s wrong with existing systems? April 9, 2013 Distributed Systems Seminar 17
Problems with existing systems � Heavy instrumentation of user and/or environment Require line of sight � � Limited range Inaccurate � High latency � Expensive � April 9, 2013 Distributed Systems Seminar 18
Prakash Prakash: Lignting Aware Motion Capture using Photosensing Markers and Multiplexed Illuminators Ramesh Raskar (Mitsubishi Electric Research Labs) et al. April 9, 2013 Distributed Systems Seminar 19
Prakash Address the following problems: Sources: http://www.vicon.com/products/cameras.html Expensive high-speed cameras required � Limited number of markers � Marker swapping � http://parasite.usc.edu/?p=403 � Special clothing and lighting required April 9, 2013 Distributed Systems Seminar 20
Prakash Main Idea: Use most basic optical devices Sources: http://de.wikipedia.org/wiki/Datei:Uvled_highres_macro.jpg https://de.wikipedia.org/wiki/Datei:Photodiode-closeup.jpg LEDs as transmitters that are fixed in the scene � Photosensors as tags to be tracked � Cheap components � April 9, 2013 Distributed Systems Seminar 21
Prakash: Receiver Tags Photosensors, a micro controller and a transmitter � Multiple photosensors used for different measurements � Sources: Prakash paper April 9, 2013 Distributed Systems Seminar 22
Prakash: Projectors � Projectors built from multiple LEDs Labelling space with binary mask � Sources: Prakash paper April 9, 2013 Distributed Systems Seminar 23
Prakash: Location Basically binary search � � Accuracy doubled by every LED 3 projectors needed for 3D location � Sources: Prakash paper and presentation April 9, 2013 Distributed Systems Seminar 24
Prakash: Orientation Multiple fixed beacons � Analog photosensor without lens � Cosine fall-off for estimation of angles � Sources: Prakash paper April 9, 2013 Distributed Systems Seminar 25
Prakash: Illumination Measure RGB illumination � � Use one photosensor per color Sources: Prakash presentation April 9, 2013 Distributed Systems Seminar 26
Prakash: Advantages IDs for markers: no swapping � � Not sensitive to lighting conditions Imperceptible tags in regular clothing � Orientation and illumination information � Faster than regular cameras � April 9, 2013 Distributed Systems Seminar 27
Prakash: Drawbacks � No solution for occlusion Wires on tags � Not suitable if motion is too fast � Simultaneity assumption: Tags don’t move while lit by projector � April 9, 2013 Distributed Systems Seminar 28
Prakash: My thoughts Elegant idea using simple means � Nice fit for filmmaking � Basically a distributed system � � How does it scale in practice? April 9, 2013 Distributed Systems Seminar 29
Body-Mounted Cameras Motion Capture from Body-Mounted Cameras Takaaki Shiratori (Disney Research) et al. April 9, 2013 Distributed Systems Seminar 30
April 9, 2013 � � Body-Mounted Cameras Address the following problems: System confined to studios Heavy instrumentation of environment Distributed Systems Seminar 31 Sources: http://www.creativeplanetnetwork.com/the_wire/2008/07/29/vicon-house-of-moves-builds- new-motion-capture-sound-stage-expands-staff-with-full-service-animation-team/
Body-Mounted Cameras Attach multiple outward looking cameras to the subject � � No instrumentation of the environment Use structure-from-motion to recover movements from camera � footage Sources: Body-Mounted Camera Paper April 9, 2013 Distributed Systems Seminar 32
Body-Mounted Cameras: Approach Sources: Body-Mounted Camera Paper April 9, 2013 Distributed Systems Seminar 33
Body-Mounted Cameras: Approach Sources: Body-Mounted Camera Paper April 9, 2013 Distributed Systems Seminar 34
Body-Mounted Cameras: Approach Sources: Body-Mounted Camera Paper April 9, 2013 Distributed Systems Seminar 35
Body-Mounted Cameras: Global Optimization � Why global optimization? Keep motion smooth and minimize reprojection error � Sources: Body-Mounted Camera Paper April 9, 2013 Distributed Systems Seminar 36
Body-Mounted Cameras: Advantages No instrumentation of the environment � Works outside: no limited range � Motion of skeleton and global root motion � 3D structure of scene as byproduct � April 9, 2013 Distributed Systems Seminar 37
Body-Mounted Cameras: Drawbacks Heavy instrumentation of user � Very long processing time � Problems with motion-blur � � Motion in the scene problematic April 9, 2013 Distributed Systems Seminar 38
Body-Mounted Cameras: My thoughts Useful as soon as cameras are significantly smaller � � Main application: biomechanics research and sports April 9, 2013 Distributed Systems Seminar 39
Humantenna Humantenna: Using the Body as an Antenna for Real-Time Whole-Body Interaction Gabe Cohn (Microsoft Research) et al. April 9, 2013 Distributed Systems Seminar 40
Humantenna Address the following problems: Heavy instrumentation of environment and/or user � High latency � Portability � Sources: Body-Mounted Camera Paper April 9, 2013 Distributed Systems Seminar 41
http://mizzoumagarchives.missouri.edu/2011-Summer/features/the-struggle-for-signal/index.php Humantenna � Body as an antenna for receiving EM noise Sources: http://en.wikipedia.org/wiki/File:Antistatic_wrist_strap.jpg No instrumentation of the environment � � Minimal instrumentation of the user � Goals: Gesture recognition � Location classification � April 9, 2013 Distributed Systems Seminar 42
Recommend
More recommend
