Phase synchronization An example of global optimality on manifolds - PowerPoint PPT Presentation

Phase synchronization An example of global optimality on manifolds Nicolas Boumal, Inria & ENS Paris Joint work with Afonso Bandeira and Amit Singer One-day workshop on Riemannian and nonsmooth optimization Sept. 25, 2015

Note for the reader A tighter version of the theorems in these slides will appear in an update to the arxiv paper 1411.3272 (version 1 from Nov. 2014 is less tight). The version in these slides was chosen as it is relatively simple to derive in the allotted time. In particular, the bound on 𝑨 − 𝑦 2 can be reduced to 𝑃(𝜏) and the tightness rate can be improved to 𝜏 ≤ 𝑑 ⋅ 𝑜 1/4 as a result. Nicolas, September 28, 2015.

The goal: estimate individual orientations, from pairwise comparisons (up to global shift)

The goal: estimate individual orientations, from pairwise comparisons (up to global shift)

Estimate phases from relative info Unknowns 𝑨 1 , … , 𝑨 𝑜 ∈ ℂ , with 𝑨 1 = ⋯ = 𝑨 𝑜 = 1 𝑜 ℂ 1 Data Noisy measurements of relative phases: ∗ + 𝜏𝑋 𝐷 𝑗𝑘 = 𝑨 𝑗 𝑨 𝑗𝑘 𝑘

What does additive Gaussian noise mean here? Noise affects the phase of the measurement ∗ + 𝜏𝑋 𝐷 𝑗𝑘 = 𝑨 𝑗 𝑨 𝑗𝑘 𝑘

Maximum likelihood estimation 𝐷 = 𝑨𝑨 ∗ + 𝜏𝑋 𝑜 , 𝑨 ∈ ℂ 1 Under Gaussian noise, the MLE solves a least-squares 2 𝐷 − 𝑦𝑦 ∗ 𝑜 𝑦 ∗ 𝐷𝑦 min ≡ max F 𝑜 𝑦∈ℂ 1 𝑦∈ℂ 1

ℓ 2 ball of radius 12𝜏 𝑜 𝑦 (the MLE) 𝑨 (the signal) level sets of 𝑦 ∗ 𝐷𝑦

Lemma: The MLE 𝑦 is close to the signal 𝑨 The MLE is more likely than the signal: 𝑨 ∗ 𝐷𝑨 ≤ 𝑦 ∗ 𝐷𝑦 . Hence, with 𝐷 = 𝑨𝑨 ∗ + 𝜏𝑋 and 𝑋 op ≤ 3 𝑜 , 𝑜 2 + 𝜏𝑨 ∗ 𝑋𝑨 ≤ 𝑨 ∗ 𝑦 2 + 𝜏𝑦 ∗ 𝑋𝑦 𝑜 2 − 𝑨 ∗ 𝑦 2 ≤ 𝜏 𝑦 ∗ 𝑋𝑦 − 𝑨 ∗ 𝑋𝑨 ≤ 𝜏 ⋅ 2𝑜 𝑋 op ≤ 6𝜏𝑜 3/2 Divide by 𝑜 + 𝑨 ∗ 𝑦 ≥ 𝑜 . Implies: 2 = 2(𝑜 − |𝑨 ∗ 𝑦|) ≤ 12𝜏 𝑜 𝑨 − 𝑦𝑓 𝑗𝜄 min 2 𝜄

The MLE is NP-hard to compute The problem 𝑜 𝑦 ∗ 𝐷𝑦 max 𝑦∈ℂ 1 has a quadratic cost 𝑦 ∗ 𝐷𝑦 , and nonconvex quadratic constraints 𝑦 𝑗 2 = 1 .

Surprisingly, global optimality can be tested (sometimes) 𝑜 , consider Given 𝑦 ∈ ℂ 1 𝑇 = 𝑇 𝑦 = ℜ ddiag 𝐷𝑦𝑦 ∗ − 𝐷 If 𝑇 is positive semidefinite, then 𝑦 is optimal: 𝑜 , 0 ≤ 𝑧 ∗ 𝑇𝑧 = 𝑦 ∗ 𝐷𝑦 − 𝑧 ∗ 𝐷𝑧 ∀𝑧 ∈ ℂ 1

Optimization on manifolds finds a certified optimum quite often Don’t know Noise Level 𝜏 Certified optimum: 𝑇 ≽ 0 Number of phases 𝑜

How is that possible? The problem is convex in a lifted space

Classic lifting trick: rewrite everything in terms of 𝑌 = 𝑦𝑦 ∗ 𝑜 𝑦 ∗ 𝐷𝑦 max 𝑦∈ℂ 1 The cost 𝑦 ∗ 𝐷𝑦 = Trace 𝑦 ∗ 𝐷𝑦 = Trace(𝐷𝑌) The constraints ∗ = 1 ⇔ 𝑌 𝑗𝑗 = 1 𝑦 𝑗 2 = 1 ⇔ 𝑦 𝑗 𝑦 𝑗 The knot ∃𝑦: 𝑌 = 𝑦𝑦 ∗ ⇔ 𝑌 ≽ 0, rank 𝑌 = 1

Suggests a semidefinite relaxation Recast the QCQP 𝑜 𝑦 ∗ 𝐷𝑦 max 𝑦∈ℂ 1 Into 𝑌∈ℂ 𝑜×𝑜 Trace(𝐷𝑌) max diag 𝑌 = 𝟐 𝑌 ≽ 0 rank 𝑌 = 1

Suggests a semidefinite relaxation Relax the QCQP 𝑜 𝑦 ∗ 𝐷𝑦 max 𝑦∈ℂ 1 Into the SDP 𝑌∈ℂ 𝑜×𝑜 Trace(𝐷𝑌) max diag 𝑌 = 𝟐 𝑌 ≽ 0 rank 𝑌 = 1

The SDP seems tight roughly when Riemannian optimization succeeds Not tight Noise Level 𝜏 Tight: SDP solution has rank 1 Number of phases 𝑜

We prove the SDP has a unique solution of rank 1 This is with high probability for large 𝑜 , And if 𝜏 ≤ 𝑑 ⋅ 𝑜 1/6 (empirically, 𝑃(𝑜 1/2 ) ok).

General idea: dual certification 𝑌∈ℂ 𝑜×𝑜 Trace(𝐷𝑌) max Lemma: diag 𝑌 = 𝟐 𝑌 ≽ 0 𝑌 solves the SDP if and only if 𝑇 𝑌 = ℜ ddiag 𝐷𝑌 − 𝐷 ≽ 0 Proof via KKT conditions. Thus, the certificate works iff the SDP is tight.

General idea: dual certification Let 𝑦 be the MLE, solution of 𝑜 𝑦 ∗ 𝐷𝑦 max 𝑦∈ℂ 1 We aim to prove that 𝑇 𝑦𝑦 ∗ ≽ 0 . Challenge : we don’t know 𝑦 .

Step 1: characterize the MLE 𝑦 𝑦 is second-order critical, close to 𝑨 . 𝑦 (the MLE) 𝑨 (the signal) level sets of 𝑦 ∗ 𝐷𝑦

Step 1: characterize the MLE 𝑦 The MLE problem lives on a smooth manifold 𝑜 𝑦 ∗ 𝐷𝑦 max 𝑦∈ℂ 1 𝑦 is critical simply if its projected gradient vanishes: 𝑦 is critical ⇔ 𝑇𝑦 = 0 𝑇 = 𝑇 𝑦 = ℜ ddiag 𝐷𝑦𝑦 ∗ − 𝐷

Step 1: characterize the MLE 𝑦 𝑇 = 𝑇 𝑦 = ℜ ddiag 𝐷𝑦𝑦 ∗ − 𝐷 𝑦 is critical ⇔ 𝑇𝑦 = 0 ⇔ diag 𝐷𝑦𝑦 ∗ is real If 𝑦 is also second-order critical, then 𝑇 is positive semidefinite on the tangent space . 𝑦 is second-order critical ⇒ diag 𝐷𝑦𝑦 ∗ ≥ 𝟐

Step 2: certify Assuming 𝑦 is second-order critical and close to 𝑨 , show that 𝑇 = ddiag 𝐷𝑦𝑦 ∗ − 𝐷 ≽ 0. For all 𝑣 ∈ ℂ 𝑜 with 𝑣 ∗ 𝑦 = 0 , seven lines give: 2 𝑜 − 𝜏 21 𝑜 + 𝑋𝑦 ∞ 𝑣 ∗ 𝑇𝑣 ≥ 𝑣 2 (Used 𝑋 op ≤ 3 𝑜 again.)

Step 2: certify Sufficient condition for tightness of the SDP: 𝑜 ≥ 𝜏 21 𝑜 + 𝑋𝑦 ∞ Note: 𝑦 and 𝑋 are not independent. Suboptimal argument (whp): 𝑋𝑦 ∞ ≤ 𝑋𝑨 ∞ + 𝑋 𝑦 − 𝑨 ∞ ≤ 𝑜 log 𝑜 + 𝑋 op 𝑦 − 𝑨 2 𝑜 log 𝑜 + 3 12𝜏𝑜 3/2 ≤

Theorem (ArXiv 1411.3272) With high probability for large (finite) 𝑜 , If 𝜏 ≤ 𝑑 ⋅ 𝑜 1/6 , Then a second-order critical point 𝑦 close to 𝑨 is optimal, with certificate 𝑇 , And 𝑦𝑦 ∗ is the unique solution of the SDP.