Invariants Risi Kondor . A classical problem Let g R , and f : R - PowerPoint PPT Presentation

Invariants Risi Kondor

. A classical problem Let g ∈ R , and f : R → C . The real numbers, as a group acts on the space of functions on R by translation: g : f �→ f ′ f ′ ( x ) = f ( x − g ) . where Question: How do we construct functionals Υ[ f ] that are invariant to this action, i.e., for which Υ[ f ] = Υ[ f ′ ] for any f and any g ? Many applications in signal processing, image analysis, etc.. 2 / 25 2/25 .

. Solution 1: Autocorrelation The autocorrelation of f is ∫ a ( x ) = f ( x + y ) f ( y ) dy. Tells us how much f changes when we translate it by an amount y . Clearly invariant to translation: 3 / 25 3/25 .

. Solution 2: Power spectrum The power spectrum of f is ∫ ∫ f ( ω ) ∗ � f ( ω ) | 2 dx. � | � � a ( ω ) = f ( ω ) dx = Literally measures the amount of energy in each Fourier mode. Clearly invariant to translation: ∫ ( e 2 πiωg � f ( ω )) ∗ ( e 2 πiωg � a f ′ ( ω ) = � f ( ω )) dx = ∫ f ( ω ) ∗ � � f ( ω ) dx = � a f ( ω ) . • In fact, the power spectrum is just the Fourier transform of the autocorrelation. • Their common limitation: lose all the information in the phase. 4 / 25 4/25 .

. 3: Triple correlation & bispectrum The triple correlation of f is ∫ b ( x 1 , x 2 ) = f ( y − x 1 ) f ( y − x 2 ) f ( y ) dy The bispectrum of f is f ( k 1 ) ∗ � f ( k 2 ) ∗ � � b ( k 1 , k 2 ) = � f ( k 1 + k 2 ) . 5 / 25 5/25 .

Reconstructing f from � . b f ( k 1 ) ∗ � f ( k 2 ) ∗ � � b ( k 1 , k 2 ) = � f ( k 1 + k 2 ) . Use the following algorithm to recover f from � b : 1. � f (0) = � b (0 , 0) 1 / 3 √ 2. � � b (0 , 1) / � f (1) = e iφ f (0) → indeterminacy in φ inevitable 3. � b (1 , k ) � f ( k + 1) = f ( k ) ∗ k = 2 , 3 , . . . f (1) ∗ � � • If � f ( k ) ̸ = 0 for any k , then � b uniquely determines � f up to translation. 6 / 25 6/25 .

Invariants on groups

. General setup • G is a group acting on a set X transitively. This means that each g ∈ G is a map g : X → X , sending g : x �→ gx. • L ( X ) is a space of functions on X . The action of G on X extends to functions in L ( X ) by g : f �→ f g ( f g )( x ) = f ( g − 1 x ) . where (Assume that gf ∈ L ( X ) for all f ∈ L ( X ) and g ∈ G .) • A functional Υ[ f ] is an invariant to this action if Υ[ f ] = Υ[ f g ] ∀ f ∈ L ( X ) , ∀ g ∈ G. 8 / 25 8/25 .

. Examples • The rotation group SO(3) acts on the sphere S 2 by x �→ Rx . Consider L 2 ( S 2 ) ... • The symmetric group acts on the adjacency matrix of a graph by ( i, j ) �→ ( σ ( i ) , σ ( j )) . 9 / 25 9/25 .

. The Fourier Transform Restrict ourselves for now to finite X and finite G . Recall that f induces a function f ↑ G ( g ) = f ( gx 0 ) , and the Fourier transform of f is ∑ � f ( ρ ) = f ( g ) ρ ( g ) ρ ∈ R G . g ∈ G Moreover, if f t ( g ) = f ( t − 1 g ) , then f t ( ρ ) = ρ ( t ) · � � f ( ρ ) . 10 / 25 10/25 .

. Noncommutative power spectrum The power spectrum of a function f : X → C is f ( ρ ) † · � a ( ρ ) = � � f ( ρ ) . Clearly invariant because f τ ( ρ ) † · � f ( ρ ) † · � � f τ ( ρ ) = ( ρ ρ ( t ) · � f ( ρ )) † ( ρ ρ ( t ) · � f ( ρ )) = � f ( ρ ) . The power spectrum is the FT of the (flipped) autocorrelation function ∑ f ( gh − 1 ) f ( g ) . a ( h ) = g ∈ G 11 / 25 11/25 .

. The noncommutative bispectrum Recall the Clebsch-Gordan decomposition [ ⊕ c ( ρ 1 ,ρ 2 ,ρ ) ] ⊕ C † ρ 1 ( σ ) ⊗ ρ 2 ( σ ) = C ρ 1 ,ρ 2 ρ ( σ ) ρ 1 ,ρ 2 . i =1 ρ ∈ R ρ 1 ,ρ 2 The bispectrum: [ ] † c ( ρ 1 ,ρ 2 ,ρ ) ⊕ ⊕ � f ( ρ 1 ) ⊗ � � � b f ( ρ 1 , ρ 2 ) = C † f ( ρ 2 ) C ρ 1 ,ρ 2 f ( ρ ) ρ 1 ,ρ 2 ρ ∈ Λ ρ 1 ,ρ 2 i =1 The bispectrum is the FT of the triple correlation ∑ f ( gh − 1 1 ) f ( gh − 1 b ( h 1 , h 2 ) = 2 ) f ( g ) . g ∈ G 12 / 25 12/25 .

. Completeness result Theorem [Kakarala, 1992]. Let f and f ′ be a pair of complex valued integrable functions on a compact group G . Assume that � f ( ρ ) is invertible for each ρ ∈ R . Then f ′ = f z for some z ∈ G if and only if b f ( ρ 1 , ρ 2 ) = b f ′ ( ρ 1 , ρ 2 ) for all ρ 1 , ρ 2 ∈ R . • Generalizes to any Tatsuuma duality group ( ISO ( n ) , ihomog. Lorentz Group, etc.) 13 / 25 13/25 .

. The skew spectrum The skew spectrum of f : S n → C is the collection of matrices r † h ( ρ ) · � � � q h ( ρ ) = � f ( ρ ) , ρ ∈ R G , ∈ G, with r h ( g ) = f ( gh ) f ( g ) . Unitarily equivalent to the bispectrum, but sometimes easier to compute [K., 2007] 14 / 25 14/25 .

Optimization problems Risi Kondor

. The Quadratic Assignment Problem Given A, A ′ ∈ R n × n , the Quadaratic Assignment Problem is to solve n ∑ A σ ( i ) ,σ ( j ) A ′ maximize f ( σ ) f ( σ ) = i,j . σ ∈ S n i,j =1 Equivalently, { 1 if σ ( j ) = i tr( P σ AP ⊤ σ A ′ ) maximize [ P σ ] i,j = 0 else . σ ∈ S n • Can interpret A and A ′ as the adjacency matrices of two (weighted) graphs. • The QAP aims to find the “best way to match” A to A ′ . • Most graph-to-graph matching problems reduce to special cases of the QAP: traveling salesman, (sub)-graph isomorphism, graph partitioning, etc. • → NP-hard. Also hard in practice. No PTAS until recently. • Sometimes written in minimization form. 16 / 25 16/25 .

. Classical approach: branch&bound • Subdivide in the form of a tree based on where vertex 1 is mapped, where vertex 2 is mapped, etc.. Each node corresponds to a coset τ i 1 ,...,i k S n − k . • Define bounds B i 1 ,...,i k ≥ σ ∈ τ i 1 ,...,ik S n − k f ( σ ) . max • Do a directed depth-first search: follow most promising branch, go down to leaf, then backtrack, but never follow branches which are guaranteed to be worse that optimum so far. • Hard to come up with theoretical performance guarantees. Empirical performance depends critically on the tightness of the bounds. 17 / 25 17/25 .

The Fourier approach to the QAP

. Fundamental Observation Consider the Fourier transform of the objective function: ∑ � f ( λ ) = f ( σ ) ρ λ ( σ ) . σ ∈ S n Theorem [Rockmore et al., 2002]. If f is the QAP objective function, then { } � f ( λ ) = 0 unless λ ∈ . , , , ( ) ( ) Proposition [K., 2010]. � and � f f are rank one matrices. Question: Why is this? 19 / 25 19/25 .

. Graph Functions S n acts transitively on the off-diagonal entries of A , so it induces a function g A ( σ ) = A σ ( n ) ,σ ( n − 1) with the property π ◦ ( g A ) = g ( π ◦ A ) (note g A ( στ ) = g A ( σ ) ∀ τ ∈ S n − 2 ). 20 / 25 20/25 .

. Graph correlation Proposition. The QAP objective function can be written in the form ∑ 1 f ( σ ) = g A ( στ ) g A ′ ( τ ) . ( n − 2)! τ ∈ S n Proof. n ∑ ∑ 1 A σ ( i ) ,σ ( j ) A ′ i,j = A σπ ( n ) ,σπ ( n − 1) A ′ π ( n ) ,π ( n − 1) = ( n − 2)! i,j =1 π ∈ S n ∑ 1 g A ( σπ ) g A ′ ( π ) . ( n − 2)! π ∈ S n 21 / 25 21/25 .

. Graph correlation Proposition. The QAP objective function can be written in the form ∑ 1 f ( σ ) = g A ( στ ) g A ′ ( τ ) . ( n − 2)! τ ∈ S n Corollary. The Fourier transform of the QAP objective is of the form 1 � g A ′ ( λ )) ⊤ , ( n − 2)! � g A ( λ ) · ( � f ( λ ) = λ ⊢ n. (1) In particular, � f ( λ ) is identically zero unless λ = ( n ) , ( n − 1 , 1) , ( n − 2 , 2) or ( n − 2 , 1 , 1) , and � f (( n − 2 , 2)) and � f (( n − 2 , 1 , 1)) are dyadic (rank one) matrices. Question: Is this useful for anything? 22 / 25 22/25 .

. Fourier space branch & bound g A ′ → O ( n 2 ) space, O ( n 3 ) time. • First compute � g A and � • Compute � f . → O ( n 2 ) space, O ( n 3 ) time. • Do branch and bound on the same coset tree as before. • Compute bounds directly in Fourier space. 23 / 25 23/25 .

. Restriction to cosets in Fourier space [ ] ∑ d λ ′ ρ λ ( in ) ⊤ · � � f i ( λ ) = f ( λ ) nλ λ ′ λ ′ ∈ λ n 24 / 25 24/25 .

. Fourier space bounds Proposition. For any f : S n → R ∑ σ ∈ S n f ( σ ) ≤ 1 d λ ∥ � max f ( λ ) ∥ tr . n ! λ ⊢ n Proof. ∑ [ � f ( λ ) · ρ λ ( σ ) − 1 ] f ( σ ) = 1 d λ tr n ! λ ⊢ n For any orthogonal matrix O , tr( MO ) ≤ ∥ M ∥ tr . • This bound is computed in time O ( n 3 ) (requires an SVD). 25 / 25 25/25 .