Solving SDPs for synchronization and MaxCut problems via the Grothendieck inequality Song Mei Stanford University July 8, 2017 Joint work with Theodor Misiakiewicz, Andrea Montanari, and Roberto I. Oliveira Song Mei (Stanford University) The landscape of non-convex SDP July 8, 2017 1 / 13
The MaxCut SDP problem ◮ ❆ ✷ R ♥ ✂ ♥ symmetric. ◮ MaxCut SDP: ♠❛①✐♠✐③❡ ❤ ❆❀ ❳ ✐ ❳ ✷ R ♥ ✂ ♥ (SDP) s✉❜❥❡❝t t♦ ❳ ✐✐ ❂ ✶ ❀ ✐ ✷ ❬ ♥ ❪ ❀ ❳ ✗ ✵ ✿ ◮ Applications: MaxCut problem, Z ✷ synchronization, Stochastic block model... Song Mei (Stanford University) The landscape of non-convex SDP July 8, 2017 2 / 13
The MaxCut SDP problem ◮ ❆ ✷ R ♥ ✂ ♥ symmetric. ◮ MaxCut SDP: ♠❛①✐♠✐③❡ ❤ ❆❀ ❳ ✐ ❳ ✷ R ♥ ✂ ♥ (SDP) s✉❜❥❡❝t t♦ ❳ ✐✐ ❂ ✶ ❀ ✐ ✷ ❬ ♥ ❪ ❀ ❳ ✗ ✵ ✿ ◮ Applications: MaxCut problem, Z ✷ synchronization, Stochastic block model... Song Mei (Stanford University) The landscape of non-convex SDP July 8, 2017 2 / 13
The MaxCut SDP problem ◮ ❆ ✷ R ♥ ✂ ♥ symmetric. ◮ MaxCut SDP: ♠❛①✐♠✐③❡ ❤ ❆❀ ❳ ✐ ❳ ✷ R ♥ ✂ ♥ (SDP) s✉❜❥❡❝t t♦ ❳ ✐✐ ❂ ✶ ❀ ✐ ✷ ❬ ♥ ❪ ❀ ❳ ✗ ✵ ✿ ◮ Applications: MaxCut problem, Z ✷ synchronization, Stochastic block model... Song Mei (Stanford University) The landscape of non-convex SDP July 8, 2017 2 / 13
MaxCut Problem ◮ ● : a positively weighted graph. ❆ ● : its adjacency matrix. ◮ MaxCut of ● : known to be NP-hard ♥ ✶ ❳ ♠❛①✐♠✐③❡ ❆ ●❀✐❥ ✭✶ � ① ✐ ① ❥ ✮ ✿ (MaxCut) ✹ ① ✷❢✝ ✶ ❣ ♥ ✐❀❥ ❂✶ ◮ SDP relaxation: ✵ ✿ ✽✼✽ -approximate guarantee [Goemanns and Williamson, 1995] ♥ ✶ ❳ ♠❛①✐♠✐③❡ ❆ ●❀✐❥ ✭✶ � ❳ ✐❥ ✮ ❀ ✹ ❳ ✷ R ♥ ✂ ♥ ✐❀❥ ❂✶ (SDPCut) s✉❜❥❡❝t t♦ ❳ ✐✐ ❂ ✶ ❀ ❳ ✗ ✵ ✿ Song Mei (Stanford University) The landscape of non-convex SDP July 8, 2017 3 / 13
MaxCut Problem ◮ ● : a positively weighted graph. ❆ ● : its adjacency matrix. ◮ MaxCut of ● : known to be NP-hard ♥ ✶ ❳ ♠❛①✐♠✐③❡ ❆ ●❀✐❥ ✭✶ � ① ✐ ① ❥ ✮ ✿ (MaxCut) ✹ ① ✷❢✝ ✶ ❣ ♥ ✐❀❥ ❂✶ ◮ SDP relaxation: ✵ ✿ ✽✼✽ -approximate guarantee [Goemanns and Williamson, 1995] ♥ ✶ ❳ ♠❛①✐♠✐③❡ ❆ ●❀✐❥ ✭✶ � ❳ ✐❥ ✮ ❀ ✹ ❳ ✷ R ♥ ✂ ♥ ✐❀❥ ❂✶ (SDPCut) s✉❜❥❡❝t t♦ ❳ ✐✐ ❂ ✶ ❀ ❳ ✗ ✵ ✿ Song Mei (Stanford University) The landscape of non-convex SDP July 8, 2017 3 / 13
MaxCut Problem ◮ ● : a positively weighted graph. ❆ ● : its adjacency matrix. ◮ MaxCut of ● : known to be NP-hard ♥ ✶ ❳ ♠❛①✐♠✐③❡ ❆ ●❀✐❥ ✭✶ � ① ✐ ① ❥ ✮ ✿ (MaxCut) ✹ ① ✷❢✝ ✶ ❣ ♥ ✐❀❥ ❂✶ ◮ SDP relaxation: ✵ ✿ ✽✼✽ -approximate guarantee [Goemanns and Williamson, 1995] ♥ ✶ ❳ ♠❛①✐♠✐③❡ ❆ ●❀✐❥ ✭✶ � ❳ ✐❥ ✮ ❀ ✹ ❳ ✷ R ♥ ✂ ♥ ✐❀❥ ❂✶ (SDPCut) s✉❜❥❡❝t t♦ ❳ ✐✐ ❂ ✶ ❀ ❳ ✗ ✵ ✿ Song Mei (Stanford University) The landscape of non-convex SDP July 8, 2017 3 / 13
Burer-Monteiro approach ◮ Convex formulation: ♥ up to ✶✵ ✸ using interior point method ♠❛①✐♠✐③❡ ❤ ❆❀ ❳ ✐ ❳ ✷ R ♥ ✂ ♥ (SDP) s✉❜❥❡❝t t♦ ❳ ✐✐ ❂ ✶ ❀ ✐ ✷ ❬ ♥ ❪ ❀ ❳ ✗ ✵ ✿ ◮ Change variable ❳ ❂ ✛ ✁ ✛ T , ✛ ✷ R ♥ ✂ ❦ , ❦ ✜ ♥ . ◮ Non-convex formulation: ♥ up to ✶✵ ✺ ♠❛①✐♠✐③❡ ❤ ✛❀ ❆✛ ✐ ✛ ✷ R ♥ ✂ ❦ ( ❦ -Ncvx-SDP) ✛ ❂ ❬ ✛ ✶ ❀ ✿ ✿ ✿ ❀ ✛ ♥ ❪ T ❀ s✉❜❥❡❝t t♦ ❦ ✛ ✐ ❦ ✷ ❂ ✶ ❀ ✐ ✷ ❬ ♥ ❪ ✿ Song Mei (Stanford University) The landscape of non-convex SDP July 8, 2017 4 / 13
Burer-Monteiro approach ◮ Convex formulation: ♥ up to ✶✵ ✸ using interior point method ♠❛①✐♠✐③❡ ❤ ❆❀ ❳ ✐ ❳ ✷ R ♥ ✂ ♥ (SDP) s✉❜❥❡❝t t♦ ❳ ✐✐ ❂ ✶ ❀ ✐ ✷ ❬ ♥ ❪ ❀ ❳ ✗ ✵ ✿ ◮ Change variable ❳ ❂ ✛ ✁ ✛ T , ✛ ✷ R ♥ ✂ ❦ , ❦ ✜ ♥ . ◮ Non-convex formulation: ♥ up to ✶✵ ✺ ♠❛①✐♠✐③❡ ❤ ✛❀ ❆✛ ✐ ✛ ✷ R ♥ ✂ ❦ ( ❦ -Ncvx-SDP) ✛ ❂ ❬ ✛ ✶ ❀ ✿ ✿ ✿ ❀ ✛ ♥ ❪ T ❀ s✉❜❥❡❝t t♦ ❦ ✛ ✐ ❦ ✷ ❂ ✶ ❀ ✐ ✷ ❬ ♥ ❪ ✿ Song Mei (Stanford University) The landscape of non-convex SDP July 8, 2017 4 / 13
Burer-Monteiro approach ◮ Convex formulation: ♥ up to ✶✵ ✸ using interior point method ♠❛①✐♠✐③❡ ❤ ❆❀ ❳ ✐ ❳ ✷ R ♥ ✂ ♥ (SDP) s✉❜❥❡❝t t♦ ❳ ✐✐ ❂ ✶ ❀ ✐ ✷ ❬ ♥ ❪ ❀ ❳ ✗ ✵ ✿ ◮ Change variable ❳ ❂ ✛ ✁ ✛ T , ✛ ✷ R ♥ ✂ ❦ , ❦ ✜ ♥ . ◮ Non-convex formulation: ♥ up to ✶✵ ✺ ♠❛①✐♠✐③❡ ❤ ✛❀ ❆✛ ✐ ✛ ✷ R ♥ ✂ ❦ ( ❦ -Ncvx-SDP) ✛ ❂ ❬ ✛ ✶ ❀ ✿ ✿ ✿ ❀ ✛ ♥ ❪ T ❀ s✉❜❥❡❝t t♦ ❦ ✛ ✐ ❦ ✷ ❂ ✶ ❀ ✐ ✷ ❬ ♥ ❪ ✿ Song Mei (Stanford University) The landscape of non-convex SDP July 8, 2017 4 / 13
Related literatures ♣ ◮ As ❦ ✕ ✷ ♥ , the global maxima of the Non Convex formulation coincide with the global maximizer of the Convex formulation [Pataki, 1998], [Barviok, 2001], [Burer and Monteiro, 2003]. ♣ ◮ As ❦ ✕ ✷ ♥ , Non Convex formulation has no spurious local maxima [Boumal, et al. , 2016]. ◮ What if ❦ remains of order ✶ , as ♥ ✦ ✶ ? Is there spurious local maxima? Sadly, yes. ◮ How is these local maxima? Empirically, they are good! Song Mei (Stanford University) The landscape of non-convex SDP July 8, 2017 5 / 13
Related literatures ♣ ◮ As ❦ ✕ ✷ ♥ , the global maxima of the Non Convex formulation coincide with the global maximizer of the Convex formulation [Pataki, 1998], [Barviok, 2001], [Burer and Monteiro, 2003]. ♣ ◮ As ❦ ✕ ✷ ♥ , Non Convex formulation has no spurious local maxima [Boumal, et al. , 2016]. ◮ What if ❦ remains of order ✶ , as ♥ ✦ ✶ ? Is there spurious local maxima? Sadly, yes. ◮ How is these local maxima? Empirically, they are good! Song Mei (Stanford University) The landscape of non-convex SDP July 8, 2017 5 / 13
Related literatures ♣ ◮ As ❦ ✕ ✷ ♥ , the global maxima of the Non Convex formulation coincide with the global maximizer of the Convex formulation [Pataki, 1998], [Barviok, 2001], [Burer and Monteiro, 2003]. ♣ ◮ As ❦ ✕ ✷ ♥ , Non Convex formulation has no spurious local maxima [Boumal, et al. , 2016]. ◮ What if ❦ remains of order ✶ , as ♥ ✦ ✶ ? Is there spurious local maxima? Sadly, yes. ◮ How is these local maxima? Empirically, they are good! Song Mei (Stanford University) The landscape of non-convex SDP July 8, 2017 5 / 13
Related literatures ♣ ◮ As ❦ ✕ ✷ ♥ , the global maxima of the Non Convex formulation coincide with the global maximizer of the Convex formulation [Pataki, 1998], [Barviok, 2001], [Burer and Monteiro, 2003]. ♣ ◮ As ❦ ✕ ✷ ♥ , Non Convex formulation has no spurious local maxima [Boumal, et al. , 2016]. ◮ What if ❦ remains of order ✶ , as ♥ ✦ ✶ ? Is there spurious local maxima? Sadly, yes. ◮ How is these local maxima? Empirically, they are good! Song Mei (Stanford University) The landscape of non-convex SDP July 8, 2017 5 / 13
Related literatures ♣ ◮ As ❦ ✕ ✷ ♥ , the global maxima of the Non Convex formulation coincide with the global maximizer of the Convex formulation [Pataki, 1998], [Barviok, 2001], [Burer and Monteiro, 2003]. ♣ ◮ As ❦ ✕ ✷ ♥ , Non Convex formulation has no spurious local maxima [Boumal, et al. , 2016]. ◮ What if ❦ remains of order ✶ , as ♥ ✦ ✶ ? Is there spurious local maxima? Sadly, yes. ◮ How is these local maxima? Empirically, they are good! Song Mei (Stanford University) The landscape of non-convex SDP July 8, 2017 5 / 13
Related literatures ♣ ◮ As ❦ ✕ ✷ ♥ , the global maxima of the Non Convex formulation coincide with the global maximizer of the Convex formulation [Pataki, 1998], [Barviok, 2001], [Burer and Monteiro, 2003]. ♣ ◮ As ❦ ✕ ✷ ♥ , Non Convex formulation has no spurious local maxima [Boumal, et al. , 2016]. ◮ What if ❦ remains of order ✶ , as ♥ ✦ ✶ ? Is there spurious local maxima? Sadly, yes. ◮ How is these local maxima? Empirically, they are good! Song Mei (Stanford University) The landscape of non-convex SDP July 8, 2017 5 / 13
Recommend
More recommend
