Early Work by DArcy Thompson Face recognition: under age variation - PDF document

Potential Projects  Recognition and classification Early Work by D’Arcy Thompson Face recognition: under age variation   Image-based disease diagnosis  Shoe image classification  Scene understanding Matching & registration   TPS-RPM registration for medical structures (shapes) Image stitching   Video analysis  Visual tracking  Action recognition  Detection Landmark point detection   License plate detection Blurred object detection  D’Arcy Thompson, 1917  Others … CIS 5543 – Computer Vision Key Points Shape Analysis  Math is helpful for morphology.  Homologous structures necessary: correspondence.  Given these, compute transformations of plane. Haibin Ling  Uses:  Nature of transformation gives clues to forces of growth.  Shapes related by simple transformation -> species are related. Many compelling examples.  Morph between species, predict intermediate species.  Can predict missing parts of skeleton. Many slides revised from D. Jacobs Shape Analysis Topics Homologies  Shape similarity  Had a long tradition  Example: known point correspondences,  Aristotle: Save only for a difference in the way of excess or defect, the parts are identical in determine similarity the case of such animals as are of one and the same genus.  Shape morphing (warping)  Example: known point correspondences,  In biology, study of homologous structures in determine warping function species preceded provided background for Darwin.  Homologous structures explained by God creating  Shape matching different species according to a common plan.  Example: determine point correspondences  Ontogeny provided clues to homology.  Combined tasks 1

Transformations  Given matching points in two images, we find a transformation of plane.  Homeomorphism (continuous, one-to-one)  This is underconstrained problem  Implicitly, seeks simple transformation.  Not well defined here, will be subject of much future research.  Intuitively pretty clear in examples considered. Logarithmically varying: eg., tapir’s toes Cannon-bone of ox, sheep, giraffe Simplest, subset of affine Smooth: amphipods (a kind of crustacean). Descriptions of shape: Clues to Growth  Somewhat different topic, shape descriptions relevant even without comparison.  Fourier descriptors  Shape context  Equal growth in all directions leads to circle (or sphere). Piecewise affine 2

Invention of Morphing?  Given transformation between species, linearly interpolate intermediate transformations.  Intermediate morphs predict intermediate species. No growth in one direction (as in a leaf on a stem), growth increases in directions away from this so r = sin(  . Pages 1070-71 Asymmetric amounts of growth on two sides. Related Species Figure 537 3

Conclusions Assumptions  Stress on homologies.  Two sets of 2D points.  Shape comparison through non-trivial  Mostly we assume there exists a correct transformations. one-to-one correspondence  Simplicity of transformation -> similarity of shape.  And this correspondence is given.  What is the simplest transformation?  This is very natural in morphometrics, where How do we find it? points are measured and labeled.  In vision we must solve for correspondence.  Transformation may leave some deviations, how are these handled? Shape Space What is shape?  Shape Spaces – Procrustes Analysis “all the geometrical information that remains when location, scale and rotational effects are filtered out from an object.” – D. G. Kendall (1984) So describe points independent of similarity  transformation.  Remove translation Simplest way: translate so point 1 is at origin, then  remove it. More elegant, translate center of mass to origin,  remove a point. Remove scale   Scale so that sum| | X i | | ^ 2 = 1. Resulting set of points is called pre-shape.   Pre because we haven’t removed rotation yet. Matching Sets of Point Features Pre-shape Find best transformation.  Notation: U and X denote sets of normalized 1. points. Points called X i and U i , with coordinates Similarity transformation, thin-plate splines. • (x i ,y i ), (u i , v i ). Measure how good it is. 2.  If we started with n points, we now have n-1 so Chamfer distance, Haussdorf distance,  that: Euclidean distance, procrustean distance, sum i= 1..n-1 x i ^ 2 + y i ^ 2 = 1. deformation energy.  So we can think of these coordinates as lying on a unit hypersphere in 2(n-1)-dimensional space. 4

Linear Pose: 2D rotation, translation & scale Shape  If we consider all possible rotations of a set of   x x . . . x         . . . cos sin 1 2 u u u t n normalized points, these trace out a closed, 1D        1 2 n s  x  y y y    curve in pre-shape space.  v v v   sin cos t   1 2 n  1 2 y  1 1 1  n  Manifold   x x . . . x     a b t 1 2 n   x    Distances between shapes can be thought of as   y y y   b a t   1 2 n  distances between these curves. y  1 1 1   Note: to compute distance, without loss of generality,     with cos , sin a s b s we can assume that one set of points (U) does not • Notice a and b can take on any values. rotate, since rotating both point sets by the same amount doesn’t change distances. • Equations linear in a, b , translation.     a s a 2 b 2 cos s • Solve exactly with 2 points, or overconstrained system with more. Procrustes Distances Similarity Matching  Full Procrustes Distance. D F  Given point sets X and U, compare by min (s,  )  U – sXR  finding similarity transformation A that  Find a scaling and rotation of X that minimizes the minimizes | | AX-U| | . Euclidean distance to U.  R(  ) means rotate by  .  X = points X 1 , … , X n  U = points U 1 , … , U n .  Partial Procrustes Distance. D P  Find A to minimize sum | | AX i – U i | | ^ 2 min   U – XR   A straightforward, linear problem.  Rotate X to minimize the Euclidean distance to U.  Taking derivatives with respect to four unknowns of A  Procrustes Distance.  gives four linear equations in four unknowns.  Rotate X to minimize the geodesic distance on the sphere from X to U. Linear Pose Solving  We can linearly find optimal similarity  Note that we now also know how to calculate the transformation that matches X to U. (ie., Full Procrustes Distance. This is just a least- minimize sum | | AX i -U i | | ^ 2, where A is a squares solution to the over-constrained problem: similarity transformation.  This is asymmetric between X and U.         . . . cos sin x x . . . x u u u       1 2   In same way we can linearly compute Full 1 2 n n s           sin cos     v v v  y y y Procrustes Distance. 1 2 1 2 n n     a b x x . . . x  This is symmetric.     1 2 n           b a y y y  Leads immediately to other procrustes distances. 1 2 n 5

Given two points on the hypersphere, we can draw the plane containing these points and the origin.  Warping Procrustes Distances is  D P = 2 sin (  / 2) D P D F = sin  • These are all monotonic in  . So D F the same choice of rotation minimizes all three. • D F is easy to compute, others are easy to compute from D F .  Why Procrustes Distance? Thin-Plate Splines  Procrustes distance is most natural. A function, f , R 2 -> R 2 is a thin-plate spline if:  Intuition: given two objects, we can produce a • Constraint: Given corresponding points: X1… Xn and U1… Un, f(Xi)= Ui. sequence of intermediate objects on a ‘straight line’ between them, so the distance between the • Energy: f minimizes the following bending energy: two objects is the sum of the distances between intermediate objects.   2 2 2          2 2 2   f f f          dxdy  This requires a geodesic.             2 2       x x y y   2 R If we think of this as the amount of bending produced by f. Allows arbitrary affine transformation. Tangent Space Thin-Plate Splines  Can compute a hyperplane tangent to the  Solution: The function f can be computed hypersphere at a point in preshape space. using straightforward linear algebra.  Project all points onto that plane.  See Principal Warps: Thin-Plate Splines and the Decomposition of Deformations, Bookstein  All distances Euclidean. Average shape  Statistical Shape Analysis, Dryden and Mardia easy to find.  This is reasonable when all shapes similar.  Extension: Can penalize mismatch of points  In this case, all distances are similar too. (using function of | | Ui – f (Xi)| | ).  Note that when  is small,  , 2sin(  / 2), sin(  )  Results: Much like D’Arcy Thompson. are all similar. 6