Near-Optimal Joint Object Matching via Convex Relaxation Yuxin - PowerPoint PPT Presentation

October 22, 2014 Near-Optimal Joint Object Matching via Convex Relaxation Yuxin Chen, Stanford University Joint Work with Qixing Huang (TTIC), Leonidas Guibas (Stanford) Page 1

Assembling Fractured Pieces Computer Assembly (Fig. credit: Huang et al 06) Manual Assembly (Ephesus, Turkey) Page 2

Structure from Motion from Internet Images Page 3

Data-Driven Shape Analysis Example: Joint Segmentation Page 4

Joint Object/Graph Matching • Given : n objects (graphs), each containing a few elements (vertices) • Goal : consistently match all similar elements across all objects Page 5

Naive Approach: Pairwise Matching • Naive Approach ◦ Compute pairwise matching across all pairs in isolation ◦ pairwise matching: extensively explored Page 6

Are Pairwise Methods Perfect? Page 7

Are Pairwise Methods Perfect? Page 7

Additional Object Helps! Page 8

Additional Object Helps! Page 9

Popular Approach: 2-Stage Method • Stage 1: Pairwise Matching ◦ Compute pairwise matching across a few pairs in isolation ◦ Use off-the-shelf pairwise methods Page 10

Popular Approach: 2-Stage Method • Stage 1: Pairwise Matching ◦ Compute pairwise matching across a few pairs in isolation ◦ Use off-the-shelf pairwise methods • Stage 2: Global Refinement ◦ Jointly refine all provided maps ◦ Criterion: exploit global consistency Page 10

Object Representation • Object ◦ a set of points ◦ drawn from the same universe • Map ◦ point-to-point correspondence Page 11

Problem Formulation • Input: a few pairwise matches computed in isolation Page 12

Problem Formulation • Input: a few pairwise matches computed in isolation • Output: a collection of maps that are ◦ close to the input matches ◦ globally consistent • NP-Hard! [Huber 02] Page 12

Prior Art spanning tree optimization detecting inconsistent spectral technique [Kim’12, [Huber’02] cycles [Zach’10, Ngu’11] Huang’12] • Pros : empirical success • Cons : ◦ little fundamental understanding (except [HuangGuibas’13]) ◦ rely on hyper-parameter tuning Page 13

Advances in Fundamental Understanding • Semidefinite Relaxation (HuangGuibas’13) : ◦ theoretical guarantees under a basic setup ◦ tolerate 50% input errors Page 14

Advances in Fundamental Understanding • Semidefinite Relaxation (HuangGuibas’13) : ◦ theoretical guarantees under a basic setup ◦ tolerate 50% input errors • Spectral Method (Pachauri et al’13) : ◦ recovery ability improves with # objects ◦ Gaussian-Wigner noise (not realistic though...) Page 14

Advances in Fundamental Understanding • Semidefinite Relaxation (HuangGuibas’13) : ◦ theoretical guarantees under a basic setup ◦ tolerate 50% input errors • Spectral Method (Pachauri et al’13) : ◦ recovery ability improves with # objects ◦ Gaussian-Wigner noise (not realistic though...) • Several important challenges remain unaddressed... Page 14

Advances in Fundamental Understanding • Semidefinite Relaxation (HuangGuibas’13) : ◦ theoretical guarantees under a basic setup ◦ tolerate 50% input errors • Spectral Method (Pachauri et al’13) : ◦ recovery ability improves with # objects ◦ Gaussian-Wigner noise (not realistic though...) • Several important challenges remain unaddressed... • Relevant problems: ◦ rotation sync (Wang et al), multiway alignment (Bandeira et al) Page 14

Challenge 1: Dense Input Errors • Input Errors ◦ A significant fraction of inputs are corrupted Ground Truth Input Maps Page 15

Challenge 1: Dense Input Errors • Input Errors ◦ A significant fraction of inputs are corrupted ◦ Prior art: — tolerate 50% input errors [HuangGuibas’2013] Ground Truth Input Maps Page 15

Challenge 2: Partial Similarity • Partial Similarity ◦ Objects might only be partially similar to each other. — e.g. restricted views at different camera positions Input Maps Subgraph Matching Page 16

Challenge 3: Incomplete Input • Partial Input Matches ◦ pairwise matching across all object pairs is — computationally expensive — sometimes inadmissible Page 17

Our Goal • Develop an effective joint recovery method ◦ strong theoretical guarantee ( address the 3 challenges ) ◦ parameter free ◦ computationally feasible tolerate dense errors handle partial similarity fill in missing matches Page 18

(Partial) Maps • One-to-one maps between (sub)-sets of elements ◦ subgraph matching / isomophism Page 19

(Partial) Maps • One-to-one maps between (sub)-sets of elements ◦ subgraph matching / isomophism • Encode the maps across 2 objects by a 0-1 matrix   0 0 1 0 X 12 := 0 0 0 1   1 0 0 0 Page 19

Matrix Representation   0 0 1 0 X 12 := 0 0 0 1   1 0 0 0 • Consider n objects Page 20

Matrix Representation   0 0 1 0 X 12 := 0 0 0 1   1 0 0 0 • Consider n objects • Matrix representation for a collection of maps   · · · I X 12 X 1 n · · · X 21 I X 2 n   X =   . . . ... . . . . . .   · · · X n 1 X n 2 I ◦ Diagonal blocks: identity matrices (self-isomophism) ◦ Sparse Page 20

Alternative Representation: Augmented Universe • All objects / sets are sub-sampled from the same universe (of size m ).   0 0 1 0 X 12 := 0 0 0 1   1 0 0 0 Page 21

Alternative Representation: Augmented Universe • All objects / sets are sub-sampled from the same universe (of size m ).   0 0 1 0 X 12 := 0 0 0 1   1 0 0 0 • Map matrix Y i between object i and the universe   0 0 0 0 1   0 0 1 0 0 1 0 0 0 0   X 12 = Y 1 Y ⊤ Y 1 := Y 2 := ⇒ , 0 0 0 1 0     2 0 0 1 0 0   0 0 0 0 1 0 0 0 1 0 � �� � � �� � m columns m columns Page 21

P.S.D. and Low-Rank Structure • Alternative Representation:     I X 12 · · · X 1 n Y 1 X 21 I · · · X 2 n Y 2     � � Y ⊤ Y ⊤ Y ⊤ X :=  = · · ·     . . . . ... . . . . 1 2 n . . . .    · · · X n 1 X n 2 I Y n � �� � m columns Page 22

P.S.D. and Low-Rank Structure • Alternative Representation:     I X 12 · · · X 1 n Y 1 X 21 I · · · X 2 n Y 2     � � Y ⊤ Y ⊤ Y ⊤ X :=  = · · ·     . . . . ... . . . . 1 2 n . . . .    · · · X n 1 X n 2 I Y n � �� � m columns • positive semidefinite and low rank: rank ( X ) ≤ m. ◦ m : universe size Page 22

Summary of Matrix Structure A consistent map matrix X 1. X � 0 = 2. low-rank 3. sparse (0-1 matrix) Y ⊤ Y X 4. X ii = I Page 23

Summary of Matrix Structure A consistent map matrix X 1. X � 0 = 2. low-rank 3. sparse (0-1 matrix) Y ⊤ Y X 4. X ii = I Input map matrix X in • a noisy version of X ⇒ — input errors • missing entries — incomplete inputs input maps X in ground truth X Page 23

Low Rank + Sparse Matrix Separation? ⇐ + additive errors: X in − X input maps: X in ground truth: X • Robust PCA / Matrix Completion? ◦ Candes et al ◦ Chandrasekahran et al minimize L , S � L � ∗ + � S � 1 , s.t. X in = + S L ⇓ (low rank) (sparse) estimate of X Page 24

Outlier Component is Highly Biased ⇐ + additive errors: X in − X input maps: X in ground truth: X • Robust PCA can handle dense corruption if ◦ the sparse component exhibits random sign patterns Page 25

Outlier Component is Highly Biased ⇐ + additive errors: X in − X input maps: X in ground truth: X • Robust PCA can handle dense corruption if ◦ the sparse component exhibits random sign patterns • Our Case? � 1 � � � · 1 X in − X m 1 · 1 ⊤ − X m 1 · 1 ⊤ − X = (1 − p true ) = p true X +(1 − p true ) E � �� � corruption rate � �� � highly biased spectral norm: (1 − p true ) n Page 25

Debias the Error Components Original Form   Y 1 Y 2 � �   Y ⊤ Y ⊤ Y ⊤ X := � 0 · · ·  .  . 1 2 n .   Y n Augmented Form   1 ⊤ � � Y 1 1 ⊤   m :=  [ 1 Y ⊤ Y ⊤ n ] � 0  Y 2  · · · 1 X 1 .  . . Y n • Equivalently, X − 1 m 11 ⊤ � 0 � �� � debiasing Page 26

Debias the Error Components Original Form   Y 1 Y 2 � �   Y ⊤ Y ⊤ Y ⊤ X := � 0 · · ·  .  . 1 2 n .   Y n Augmented Form   1 ⊤ � � Y 1 1 ⊤   m :=  [ 1 Y ⊤ Y ⊤ n ] � 0  Y 2  · · · 1 X 1 .  . . Y n • Equivalently, X − 1 m 11 ⊤ � 0 � �� � debiasing � m 11 ⊤ � X − 1 • rank = rank ( X ) − 1 ⇒ one more degree of freedom Page 26

Objective Function X ≥ 0 , X � 0 • Ecourage consistency with provided maps � X , X in � ( to maximize ) Page 27

Objective Function X ≥ 0 , X � 0 • Ecourage consistency with provided maps � X , X in � ( to maximize ) • Promote Sparsity � X � 1 = � X , 11 ⊤ � ( to minimize ) Page 27