The Paulsen problem, continuous operator scaling, and smoothed analysis Lap Chi Lau, University of Waterloo Joint work with Tsz Chiu Kwok (Waterloo), Yin Tat Lee (Washington), Akshay Ramachandran (Waterloo)
Outline Pa Part I: Pa Paulsen problem - Motivation from frame theory Pa Part II: Continuous operator scaling - Operator scaling, alternating algorithm, reduction - Analysis of dynamical system Pa Part III: Smoothed analysis - Proof outline, capacity lower bound Part IV: Discussions Pa 2
Frames e: a collection of vectors ! " , ! $ , … , ! & ∈ ℝ ) that spans ℝ ) Fr Fram ame: Eq Equal norm : if ! * $ = ! , $ for all -, .. 2 = 3 ) . & : if ∑ *1" Parseval: Pa ! * ! * An equal norm Parseval frame is an overcomplete basis: & ∀ 5 ∈ ℝ ) 4 5, ! * ! * = 5 *1" It has applications in signal processing, communication theory, and quantum information theory. 3
Motivation Equal norm Parseval frames are difficult to construct with only a few known algebraic constructions. [Holmes-Paulsen 04] were interested in constructing Grassmaniann frames, ( , equal norm Parseval frames with minimal max ' $ , ' & $,& which are even more difficult to construct. It is easier to construct “approximate” equal norm Parseval frames (e.g. random unit vectors, optimal packing of lines). Qu Question : Can we turn an “approximate” frame into an equal norm Parseval frame by just moving the vectors “slightly”? 4
The Paulsen Problem What is the best function ! ", $, % such that for any & ' , … , & ) ∈ ℝ , with 1 − % $ 1 ≤ 1 + % $ " ≤ & 0 ∀ 1 ≤ 4 ≤ " (ϵ − nearly equal norm) 1 " ) F ≼ 1 + % B , 1 − % B , ≼ D & 0 & 0 (ϵ − nearly Parseval), 0E' there exist J ' , … , J ) ∈ ℝ , with ) 1 = $ F = B , J 0 ∀ 1 ≤ 4 ≤ " and D J 0 J 0 1 " 0E' such that ) 1 ≤ ! ", $, % ? D & 0 − J 0 1 5 0E'
Previous work [Bodmann-Casazza, 10] ! ", $, % ≤ ' " () $ *+ % ) when gcd ", $ = 1. dynamical system improves on equal norm while keeping Parseval. • [Casazza-Fickus-Mixon, 12] ! ", $, % ≤ ' " )2/4 $ )/4 % )/4 gradient descent improves on Parseval while keeping equal norm. • There are examples showing that ! ", $, % ≥ "%. Qu Question : Can the bound be independent of $ ? 6
Main Result m. ! ", $, % ≤ ' " ()/+ % Th Theorem. The proof has two parts. First, we define a dynamical system based on operator scaling, and show that ! ", $, % ≤ ' " + $ % . Then, we do a smoothed analysis to remove the dependency on $ . *[Hamilton, Moitra 18] ! ", $, % ≤ ' " + % 7
Outline Pa Part I: Pa Paulsen problem - Motivation from frame theory Pa Part II: Continuous operator scaling - Operator scaling, alternating algorithm, reduction - Analysis of dynamical system Pa Part III: Smoothed analysis - Proof outline, capacity lower bound Part IV: Discussions Pa 8
Alternating Algorithm How to move an approximate frame to satisfy the two conditions exactly? The problems is difficult with two conditions. It is easy with one condition. To satisfy the equal norm condition, we just rescale the vectors. • To satisfy the Parseval condition, we can set • +' ( ( , ) = 3 4 . ) ) ! " ← (% ! " ! " ! " so that % ! " ! " "&' "&' A natural algorithm is to alternate between these two steps and hope that it will converge to a solution satisfying both conditions. 9
First Idea Our starting point is to bound the distance by the total movement in the alternating algorithm (assuming it converges): $ } {" # ' } {" # & } {" # ( } … {" # This is a special case of the alternating algorithm for operator scaling, which was analyzed in [Gurvits 04, Garg-Gurvits-Oliveira-Wigderson 16]. 10
Operator Scaling An operator is a collection of matrices ! " , … , ! % ∈ ℝ (×* . [Gurvits 04] Given ! " , … , ! % ∈ ℝ (×* , we would like to find + ∈ ℝ (×( and , ∈ ℝ *×* such that if we define - . = +! . , for 1 ≤ 2 ≤ 3 then % % 6 = 738 ( 6 - 4 - . - and 4 - . = 7<8 * . . .5" .5" for some constant c. We say an operator satisfying the two conditions do doubl ubly ba balanc nced . 11
Alternating Algorithm Repeat the following two steps [Gurvits 04]: ' = ) * , we set % To satisfy the condition ∑ "#$ & " & " • 0$ % 1 ' ) & " ← (- & . & & " . .#$ % ' & " = ) 2 , we set To satisfy the condition ∑ "#$ & " • 0$ % 1 ' & & " ← & " (- & . ) . .#$ A natural algorithm is to alternate between these two steps and hope that it will converge to a solution satisfying both conditions. 12
Reduction A simple reduction from frame scaling to operator scaling: | | | ! " ∈ ℝ % ∈ ℝ %×, 0 ! " 0 → ' " ≡ | | | 0 = 2 % is the Parseval condition ∑ "./ 0 = 2 % . , , The condition ∑ "./ ' " ' " ! " ! " • % , 0 ' " = The condition ∑ "./ ' " , 2 , is the equal norm condition • 3 ! / 3 ⋯ 0 = 7 8 2 , . ⋮ ⋱ ⋮ 3 0 ⋯ ! , 3 So we focus on this more general setting in this part of the talk. 13
The Operator Paulsen Problem What is the best function ! ", $, %, & s.t. for any ' ( , … , ' * ∈ ℝ -×/ with * * 1 − & " 7 ' 5 ≼ 1 + & " 7 ≼ 1 + & 2 - , 1 − & 2 - ≼ 4 ' 5 ' 5 $ 2 / ≼ 4 ' 5 $ 2 / 56( 56( * ∈ ℝ -×/ with there exist 9 ( , … , 9 * * 5 = " 7 = 2 - 7 9 4 9 5 9 and 4 9 $ 2 / 5 5 56( 56( such that * ? ≤ ! ", $, %, & ≤ " ? $& 4 ' 5 − 9 ? 5 > 56( 14
Applications Ma Matri rix Sc Scaling: Preconditioning for linear solvers [Osborne 60] • Optimal transportation [Wilson 69] • Bipartite matching • Deterministic approximation of permanents [Linial-Samorodnitsky-Wigderson 00] • Fr Fram ame e Scalin aling: Sign rank lower bound [Forster 02] • Robust subspace recovery [Hardt-Moitra 13] • Paulsen problem • PS PSD scaling: Approximation of mixed discriminants [Gurvits-Samorodnitsky 02] • Op Operator Scaling: Computing non-commutative rank [Garg-Gurvits-Oliveira-Wigderson 16] • Computing Brascamp-Lieb constants [Garg-Gurvits-Oliveira-Wigderson 17] • Orbit intersection problem [AllenZhu-Garg-Li-Oliveira-Wigderson 18] • 15
Issues in First Idea * } {( ) - } {( ) , } {( ) . } … {( ) There are examples which do not converge: 0 , 1 1 0 , 0 , 0 2/2 2/2 ⇔ , 1 1 0 0 Even if it converges, the path could zig-zag a lot and the total movement is much larger than the distance. 16
Error Measure [Gurvits 04] * * ' ' Δ = 1 + 1 7 7 ( !2 3 − - 5 ( 6 ( !2 9 − / 5 ( 6 6 6 - / 6%& 6%& ) ) * is the size of the operator. ' where ! = ∑ $%& ( $ ) Δ is zero if and only if the operator is doubly balanced. • Can show that Δ ≤ - * . * . • Focus on proving the total movement is ≤ -/ Δ ≤ - * /ϵ . • The dynamical system is moving in the direction that minimizes Δ . 17
Continuous Operator Scaling Dynamical System em: Do both steps simultaneously and continuously. 0 0 ! 1 ) # $ + # $ ('( 4 − 5 , 1 # !" # $ = ('( ) − + , # - # # - ) - - -./ -./ 8 is the si 0 where ' = ∑ $./ # $ size of the operator. 7 We find some nice identities to analyze the convergence. 0 8 ! ! ! !" Δ 9 = − , !" ' 9 = −Δ 9 . 9 Le Lemma 2 2. Lemma 1 Le 1. !" # $ . 7 $./ Claim. Cl m. The dynamical system converges to a doubly balanced operator. 18
Total Movement & } $ } {" # {" # ' } {" # We again bound the final distance by the path length. * * distance + + . ' 1 . . . ' − " # & 12 $ ( " # = ( 0 12 " # - $ - #)* #)* * local + . . ' ' − 1 1 movement 12 Δ & 12 & = 0 ≤ 0 ( 12 " # 12 - $ $ #)* (Lemma 2) (triangle inequality) 19
Half Time Let ! be the first time that Δ # = Δ % /2. . # # # − + − + ≤ ! Δ % . +, Δ - +, +, Δ - +, ≤ ) 1 +, ) ) % % % We can complete the movement bound by a geometric sum argument. So it remains to bound the ha half time. Note Lemma 1 implies for all time up to T: + +, 1 - = −Δ - ≤ −Δ % /2 20
Capacity [Gurvits 04] Potential function to analyze operator scaling 9 : < = 2 345 ∑ 789 ; 7 *; 7 cap $ % = inf 9 *∈ℝ -×- ,*≻1 345 * - Le Lemma 3 3. Capacity is unchanged over time . > ? ≥ cap ? ≥ > ? − BC Δ ? . Lemma 4 Le 4. We adapt the proof of Lemma 4 from [GGOW 16]. One implication is that > F = cap F = cap 1 . 21
Bounding Half Time time. Want to upper bound the first time ! so that Δ # = Δ % /2. Half Half tim Le Lemma 3 3. Capacity is unchanged over time . ) * ≥ cap * ≥ ) * − 01 Δ * . Le Lemma 4 4. ) # ≥ cap # = cap % ≥ ) % − 01 Δ % size of the operator decreases by at most 01 Δ % 2 23 ) * = −Δ * . 5 Δ % ! 4 Le Lemma 1 1. size decreases by at least ! ≤ 201 total movement ≤ !Δ ≤ 01 Δ. Δ 22
Recommend
More recommend
