Motion capture Captures the subtlety of human motion Keyframing - PowerPoint PPT Presentation

Motion capture • Captures the subtlety of human motion • Keyframing difficult and not as successful • Cons: – We can use only prerecorded material (with certain adaptations)-limiting – Data valid only for same scale avatar-need for retargeting – How to satisfy constraints

Applications of MoCap • TV and film • Computer games (85%): no need for top quality motion data – Motion of points on the surface does not lead to successful motion of skeleton • Virtual reality (control of avatars, autonomous avatars)

Low level processing • Requires effort • Noise removal • Filling the gaps when marker is lost • Eliminating confusion when markers coincide • Conversion from positional data to joints rotations for driving a skeleton

• Storage of a vocabulary of sequences • Blending and adaptation in real time to generate new sequences • Time consuming ops like retargeting off line.

Rotation around the three local coordinate axes

The Biovision BVH • Hierarchical representation of skeleton • Translation-rotation to the root node • Rotation to other nodes (joints) • Offset between nodes (link dimension)

MoCap data editing • MoCap data can be manipulated up to a certain point before their integrity and naturalness becomes compromised. • If editing does not provide the desired sequence then we should capture a new sequence.

Changing the speed of motion • K>1 speeding up (decimation) • K<1 slowing down (interpolation) • Speeding up might also require interpolation if k non-integer

Changing the speed of motion

Changing the speed of motion

Changing the speed of motion

Changing the speed of motion

Blending • Usually to provide transition between sequences • Fade in/out parts of two sequences

Blending • For blending sequences represented as B- splines, simply blend control points • Blending has no theoretical validity • Blending a run and a walk might not generate a valid/realistic transition • Works well when sequences are aligned and similar

Blending • Similar: different manifestations of the same action e.g. walk • Aligned key poses (e.g. foot on ground) occurring at the same time. • Need for non-uniform re-sampling: time warping

Time warping

Multi-target interpolation • Blending two sequences (all over their length) to create a new sequence – Interpolation between a tired and brisk walking sequence -> sequence with both flavours. • Sequences need to be aligned

No alignment: uncoordinated motion, feet might not even touch ground

Time warping • Warping of sequences can create new sequences without blending • Warp between brisk walk and drunken, slow walk – Drunken->brisk = drunken walk at brisk pace – Brisk->drunken = brisk walk at drunken pace

Time warping • Difficult step: how to find alignment automatically • It might be impossible/inapropriate to find a suitable time warp – Complex motions (gymnastics), completely different motions • Blending after warping in such cases will create unfeasible / unrealistic motion

Time warping • Similarities with contour triangulation & speech recognition • Sequences A and B of length n • To each vertex of one signal, assign (at least) one vertex in the other signal so as to minimize a cost function measuring their “difference” • Try different correspondences between points in A and B, use dynamic programming

Time warping • Finding correspondences is equivalent to finding an optimal path on a 2D grid

Time warping • Cost: how much work it takes to deform one signal into the other. • Sum of local stretching and bending – Stretching: difference in distances between two consecutive samples on the two curves – Bending: difference in angle between three consecutive samples on the two curves

Time warping

Time warping • Once correspondences are found actual warping should be performed (B to A) • substitution: 1:1 correspondence of successive samples. (diagonal line) • deletion: multiple samples of B map to a sample of A. (horizontal line) • insertion: a sample of B maps to multiple samples of A (vertical line)

Time warping

Time warping • Substitution: B j corresponds to A i . – B wi = B j . • Deletion ( B j ,B j+ 1 ,…B j+k ) correspond to A i , – B wi = mean ( B j ,B j+ 1 ,…B j+k ) • Insertion B j corresponds to ( A i ,A i+ 1 ,…A i+k ) – ( B wi ,B wi+ 1 ,…B wi+k ) evaluated by fitting a spline around B j

Motion warping • Making local adjustments to the amplitude of motion data – Alter walking motion so that character bends to walk through a doorway – Increase the height of a jump to clear new obstacle • Use of a displacement function / map • f’(t) = f(t) + d(t)

Motion warping • Must interpolate to ‘spread’ the effect

Motion warping • For real time motion control: – Predict interaction with object or obstacle – Pick up appropriate displacement map depending on object/obstacle position – Compose new motion

Blending: Verbs and Adverbs • Extension of blending between two sequences. • Verb: a certain type of motion e.g. walk • Take a number of example motions from the verb and interpolate to obtain a continuous space of motions in this verb

Blending: Verbs and Adverbs • Parametrize verb by adverbs (adverb space) – Happy-sad, knowledgeable-clueless • Use sparse data interpolation methods (e.g. using RBFs) over the control points of the examples • For a certain frame i use p i for all examples to fit a continuous function (combination of RBFs)

Blending: Verbs and Adverbs • Examine recorded motions and place them in appropriate positions on the adverb plane according to their characteristics • Examples should be time aligned (time- warped): similar key poses to occur at the same time.

MoCap signal processing • MoCap data are 1-D discrete time data, samples of a continuous signal • Classic 1-D signal processing principles apply • However MoCap data are angular (Euler angles) • Classical theory for data on the line does not apply in the strict sense. • Acceptable if small movements occur

MoCap signal processing • Aliasing: sampling frequency must be at least twice the largest frequency of the signal – Lowpass filtering • Problems in change of speed operations

MoCap signal processing • Data have a finite length, start and end abruptly • This introduces high frequencies to the signal: leakage • A data weighting function or a non- rectangular window should be used on the MoCap data

MoCap signal processing • Hanning window • w(t)=cos 2 ( π t/T 0 ), |t|<=T 0 /2 • Windowing is also needed in concatenation of a motion sequence to create a cyclic (repeated) motion when the start/end of the sequence does not match – High frequencies are introduced if no windowing is used.

MoCap signal processing • Filtering with appropriate filters can change the character of the motion • Walking sequence – Increase middle freqs: smoothed but exaggerated – Increase high freqs: nervous, jerky – Increase low freqs: attenuated, reduced joint movement

Constraint based approaches • Change mocap data in order to satisfy constraints • Different from what we had so far • Motion fitting or re-targeting

Constraint based approaches • Adapt a sequence to a character of different scale • Fit “switch on light” data to a different character – Joint angles have to be changed in a non-linear manner – Some angles might not have to be changed at all – Possible collision detection

Constraint based approaches • Adapt a sequence to make a new sequence • Adapt a “grasp object” sequence to “grasp a different object” sequence for the same character – Can use collision detection and IK to modify MoCap data and prevent penetration

Kinematic constraints approach • Joint limits, end effector positions • Retargetting a “turn on switch” motion to another character. • Basic question: what is more important to be maintained end effector positions or joint angles?

Kinematic constraints approach • Assign importance metric to different constraints – Root position is usually not important – The position of end effector is important if it is interacting with an object – The position of end effector is important if it is near the object, moving towards the object • Use of heuristic rules to specify importance • Change aspects that are not important.

Dynamic constraints approach • Space-time constraints • Use forces and apply constraints on them • Minimize some energy function • Evaluate the trajectories