Lecture 26: MDS / Canonical Forms COMPSCI/MATH 290-04 Chris Tralie, - PowerPoint PPT Presentation

Lecture 26: MDS / Canonical Forms COMPSCI/MATH 290-04 Chris Tralie, Duke University 4/19/2016 COMPSCI/MATH 290-04 Lecture 26: MDS / Canonical Forms

Announcements ⊲ Group Assignment 3 Final Deadline Tuesday 4/26 ⊲ Guest Lecture Thursday ⊲ No office hours Thursday COMPSCI/MATH 290-04 Lecture 26: MDS / Canonical Forms

Spin Images Why did they all look so boring and unlike the objects in question? COMPSCI/MATH 290-04 Lecture 26: MDS / Canonical Forms

Spin Images I made a mistake on the assignment! First principal axis is vertical axis in image COMPSCI/MATH 290-04 Lecture 26: MDS / Canonical Forms

Table of Contents ◮ Multidimensional Scaling ⊲ Canonical Forms COMPSCI/MATH 290-04 Lecture 26: MDS / Canonical Forms

Academic Majors Distances: Your Choices Multidimensional Scaling down to R 2 0.3 0.2 Art History 0.1 ECE CS 0 English -0.1 Philosophy Math -0.2 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 COMPSCI/MATH 290-04 Lecture 26: MDS / Canonical Forms

Academic Majors Distances: Chris’s Choices Multidimensional Scaling down to R 2 0.3 0.2 Art History ECE 0.1 CS 0 -0.1 English Philosophy Math -0.2 -0.3 -0.4 -0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 COMPSCI/MATH 290-04 Lecture 26: MDS / Canonical Forms

Multidimensional Scaling ⊲ Given an N × N symmetric discrete similarity matrix D (i.e. D ij = D ji ) ⊲ Given a Euclidean dimension K Find a point cloud X ∈ R N × K so that � K � � � ( X [ i , k ] − X [ j , k ]) 2 D ij ≈ � k = 1 COMPSCI/MATH 290-04 Lecture 26: MDS / Canonical Forms

Multidimensional Scaling ⊲ Given an N × N symmetric discrete similarity matrix D (i.e. D ij = D ji ) ⊲ Given a Euclidean dimension K Find a point cloud X ∈ R N × K so that � K � � � ( X [ i , k ] − X [ j , k ]) 2 D ij ≈ � k = 1 In other words, find a point cloud in Euclidean K -space that best approximates the distances COMPSCI/MATH 290-04 Lecture 26: MDS / Canonical Forms

MDS: Euclidean Dimension Reduction � K � � � ( X [ i , k ] − X [ j , k ]) 2 D ij ≈ � k = 1 What if D ij comes from a Euclidena space of dimension d > k ? Can we solve this using something else we learned in the course? COMPSCI/MATH 290-04 Lecture 26: MDS / Canonical Forms

MDS: Euclidean Dimension Reduction � K � � � ( X [ i , k ] − X [ j , k ]) 2 D ij ≈ � k = 1 What if D ij comes from a Euclidena space of dimension d > k ? Can we solve this using something else we learned in the course? This is equivalent to PCA!! If we let k = d , then we can represent distances exactly COMPSCI/MATH 290-04 Lecture 26: MDS / Canonical Forms

MDS: Non-Euclidean Space Reduction Can we always find a point cloud that satisfies a given D by making k arbitrarily high? COMPSCI/MATH 290-04 Lecture 26: MDS / Canonical Forms

MDS: Non-Euclidean Space Reduction Can we always find a point cloud that satisfies a given D by making k arbitrarily high? Assume sphere of radius 2 /π with points in the following configuration: 4 v 1 v 2 v 3 v 4 v 1 v 2 3 v 3 2 1 v 4 COMPSCI/MATH 290-04 Lecture 26: MDS / Canonical Forms

MDS: Non-Euclidean Space Reduction Can we always find a point cloud that satisfies a given D by making k arbitrarily high? Assume sphere of radius 2 /π with points in the following configuration: 4 v 1 v 2 v 3 v 4 v 1 0 2 1 1 v 2 2 0 1 1 3 v 3 1 1 0 1 2 1 v 4 1 1 1 0 COMPSCI/MATH 290-04 Lecture 26: MDS / Canonical Forms

MDS: Non-Euclidean Space Reduction Assume sphere of radius 2 /π with points in the following configuration: 4 v 1 v 2 v 3 v 4 v 1 0 2 1 1 3 v 2 2 0 1 1 2 1 v 3 1 1 0 1 v 4 1 1 1 0 COMPSCI/MATH 290-04 Lecture 26: MDS / Canonical Forms

MDS: Non-Euclidean Space Reduction Assume sphere of radius 2 /π with points in the following configuration: 4 v 1 v 2 v 3 v 4 v 1 0 2 1 1 3 v 2 2 0 1 1 2 1 v 3 1 1 0 1 v 4 1 1 1 0 2 1 1 v1 v3 v2 v 1 , v 3 , v 2 along a line COMPSCI/MATH 290-04 Lecture 26: MDS / Canonical Forms

MDS: Non-Euclidean Space Reduction Assume sphere of radius 2 /π with points in the following configuration: 4 v 1 v 2 v 3 v 4 v 1 0 2 1 1 3 v 2 2 0 1 1 2 1 v 3 1 1 0 1 v 4 1 1 1 0 2 2 1 1 1 1 v1 v3 v2 v1 v4 v2 v 1 , v 4 , v 2 also along v 1 , v 3 , v 2 along a line line! COMPSCI/MATH 290-04 Lecture 26: MDS / Canonical Forms

MDS: Non-Euclidean Space Reduction 2 2 1 1 1 1 v1 v3 v2 v1 v4 v2 v 1 , v 4 , v 2 also along v 1 , v 3 , v 2 along a line line! This implies that v 4 and v 3 must collapse to the same point in any Euclidean space. ⊲ In other words, distances along the sphere cannot be perfectly realized using a Euclidean space of any finite dimension! COMPSCI/MATH 290-04 Lecture 26: MDS / Canonical Forms

MDS: Non-Euclidean Space Reduction 2 2 1 1 1 1 v1 v3 v2 v1 v4 v2 v 1 , v 4 , v 2 also along v 1 , v 3 , v 2 along a line line! This implies that v 4 and v 3 must collapse to the same point in any Euclidean space. ⊲ In other words, distances along the sphere cannot be perfectly realized using a Euclidean space of any finite dimension! ⊲ (But let’s do our best and see what we come up with) COMPSCI/MATH 290-04 Lecture 26: MDS / Canonical Forms

Table of Contents ◮ Multidimensional Scaling ⊲ Canonical Forms COMPSCI/MATH 290-04 Lecture 26: MDS / Canonical Forms

Nonrigid Shape Alignment How do I align these two camels?? COMPSCI/MATH 290-04 Lecture 26: MDS / Canonical Forms

Geodesic Distances Geodesic distances are invariant to isometries (aka bending without stretching) COMPSCI/MATH 290-04 Lecture 26: MDS / Canonical Forms

Geodesic Distances Geodesic distances are invariant to isometries (aka bending without stretching) What if we try to apply MDS to the distance matrix we get from geodesic distances? COMPSCI/MATH 290-04 Lecture 26: MDS / Canonical Forms

Face Example COMPSCI/MATH 290-04 Lecture 26: MDS / Canonical Forms

Face Example COMPSCI/MATH 290-04 Lecture 26: MDS / Canonical Forms

Face Example COMPSCI/MATH 290-04 Lecture 26: MDS / Canonical Forms

Face Example COMPSCI/MATH 290-04 Lecture 26: MDS / Canonical Forms

More Examples Bronstein COMPSCI/MATH 290-04 Lecture 26: MDS / Canonical Forms

Lots More Examples Elad Kimmel 2001: “Bending Invariant Representations for Surfaces” COMPSCI/MATH 290-04 Lecture 26: MDS / Canonical Forms