Overview Motivation and Introduction Solving CMPs A heuristic - PowerPoint PPT Presentation

Solving complex-valued ℓ 0 minimization problems with constant modulus constraints Frederic Matter Joint work with Tobias Fischer (Fraunhofer ITWM Kaiserslautern), Ganapati Hegde (TU Darmstadt), Marius Pesavento (TU Darmstadt), Marc Pfetsch (TU Darmstadt) and Andreas Tillmann (RWTH Aachen) March 07, 2018 | SCIP Workshop, Aachen | Frederic Matter | 1

Overview Motivation and Introduction Solving CMPs A heuristic Application Implementation Conclusion and Outline March 07, 2018 | SCIP Workshop, Aachen | Frederic Matter | 2

Introduction We want to solve the following problem Constant Modulus Program (CMP) x ∈ C N � x � 0 min √ s . t . � b − Ax � 2 ≤ δ , | x n | ∈ { 0, c } , ∀ n ∈ [ N ], with A ∈ C K × N , b ∈ C K , δ ∈ R , c ∈ R , [ N ] = { 1, ... , n } and � x � 0 denoting the number of nonzero entries in x . ◮ In the problem we could also allow additional constraints, such as linear (in-)equalities. ◮ Throughout this talk, we assume w.l.o.g. that c = 1. March 07, 2018 | SCIP Workshop, Aachen | Frederic Matter | 3

Motivation ◮ Generalization of compressed sensing: search sparse solutions of an optimization problem ⇒ use ℓ 0 -term in objective function. ◮ Note that the constant modulus constraint is nonconvex. ◮ We want to solve this problem exactly. ◮ constant modulus often appears in signal processing (e.g., phase retrieval). March 07, 2018 | SCIP Workshop, Aachen | Frederic Matter | 4

MINLP–Reformulation ◮ auxiliary binary variables y = [ y 1 , y 2 , ... , y N ] T ∈ { 0, 1 } N . ◮ w n = Re[ x n ] and z n = Im[ x n ] as real and imaginary part of x n , respectively. ◮ w = [ w 1 , w 2 , ... , w N ] T and z = [ z 1 , z 2 , ... , z N ] T . ◮ A T = [ a 1 , ... , a K ] ∈ C N × K . March 07, 2018 | SCIP Workshop, Aachen | Frederic Matter | 5

MINLP–Reformulation ◮ auxiliary binary variables y = [ y 1 , y 2 , ... , y N ] T ∈ { 0, 1 } N . ◮ w n = Re[ x n ] and z n = Im[ x n ] as real and imaginary part of x n , respectively. ◮ w = [ w 1 , w 2 , ... , w N ] T and z = [ z 1 , z 2 , ... , z N ] T . ◮ A T = [ a 1 , ... , a K ] ∈ C N × K . N K � � 2 � Re[ a k ] T w − Im[ a k ] T z � � � min y n s.t. Re[ b k ] − w , z ∈ R N , y n =1 k =1 � � 2 � Re[ a k ] T z + Im[ a k ] T w � Im[ b k ] − ≤ δ , + w 2 n + z 2 n ≤ y n , ∀ n ∈ [ N ], w 2 n + z 2 n ≥ y n , ∀ n ∈ [ N ], y n ∈ { 0, 1 } , ∀ n ∈ [ N ]. March 07, 2018 | SCIP Workshop, Aachen | Frederic Matter | 5

Overview Motivation and Introduction Solving CMPs A heuristic Application Implementation Conclusion and Outline March 07, 2018 | SCIP Workshop, Aachen | Frederic Matter | 6

Solving the resulting MINLP ◮ Apply spatial branch-and-cut framework. ◮ Error bound ( ℓ 2 -norm) constraint is convex, even in the complex case, as it can be recast as a second-order cone constraint. [ x 1 , ... , x k , x k +1 ] T ∈ R k +1 : � [ x 1 , ... , x k ] T � 2 ≤ x k +1 � � second-order cone: C k = . second-order cone constraint: � A i x + b i � 2 ≤ c T i x + d i , A i ∈ R ( n i − 1) × n , b i ∈ R n i − 1 , c i ∈ R n , d i ∈ R . ◮ Upper bound parts of modulus constraints are convex quadratic constraints. ◮ Lower bound parts of modulus constraints are nonconvex quadratic constraints. March 07, 2018 | SCIP Workshop, Aachen | Frederic Matter | 7

Solving the resulting MINLP ◮ Apply spatial branch-and-cut framework. ◮ Error bound ( ℓ 2 -norm) constraint is convex, even in the complex case, as it can be recast as a second-order cone constraint. [ x 1 , ... , x k , x k +1 ] T ∈ R k +1 : � [ x 1 , ... , x k ] T � 2 ≤ x k +1 � � second-order cone: C k = . second-order cone constraint: � A i x + b i � 2 ≤ c T i x + d i , A i ∈ R ( n i − 1) × n , b i ∈ R n i − 1 , c i ∈ R n , d i ∈ R . ◮ Upper bound parts of modulus constraints are convex quadratic constraints. ◮ Lower bound parts of modulus constraints are nonconvex quadratic constraints. ⇒ exploit structure of modulus constraints to develop specialized branching and propagation (and some very easy separation) to enforce modulus constraints. March 07, 2018 | SCIP Workshop, Aachen | Frederic Matter | 7

The LP relaxation Another problem: poor linear relaxation. ⇒ Use additional linear inequalities that strengthen the LP relaxation. March 07, 2018 | SCIP Workshop, Aachen | Frederic Matter | 8

The LP relaxation Another problem: poor linear relaxation. ⇒ Use additional linear inequalities that strengthen the LP relaxation. √ √ z n w n + z n ≤ − w n + z n ≤ 2 y n 2 y n − y n ≤ w n ≤ y n , w n ≥ − y n w n ≤ y n − y n ≤ z n ≤ y n , √ z n ≤ y n w n + z n ≤ 2 y n , √ w n − z n ≤ 2 y n , w n √ − w n + z n ≤ 2 y n , √ z n ≥ − y n − w n − z n ≤ 2 y n . √ √ w n − z n ≤ − w n − z n ≤ 2 y n 2 y n March 07, 2018 | SCIP Workshop, Aachen | Frederic Matter | 8

Enforcing Modulus Constraints y ) of the LP relaxation violates w 2 n + z 2 If the solution ( ˆ w , ˆ z , ˆ n ≥ y n for some n ∈ [ N ]: z n ( ˆ w n , ˆ z n ) w n March 07, 2018 | SCIP Workshop, Aachen | Frederic Matter | 9

Enforcing Modulus Constraints y ) of the LP relaxation violates w 2 n + z 2 If the solution ( ˆ w , ˆ z , ˆ n ≥ y n for some n ∈ [ N ]: ◮ If binary variable ˆ y n fixed to zero: set ˆ w n , ˆ z n to zero as well. z n ( ˆ w n , ˆ z n ) w n March 07, 2018 | SCIP Workshop, Aachen | Frederic Matter | 9

Enforcing Modulus Constraints y ) of the LP relaxation violates w 2 n + z 2 If the solution ( ˆ w , ˆ z , ˆ n ≥ y n for some n ∈ [ N ]: ◮ If binary variable ˆ y n fixed to zero: set ˆ w n , ˆ z n to zero as well. ◮ If bounds of the continuous variables w n and z n are not yet restricted to one of the four orthants: create four branching nodes, one for each orthant. z n ( ˆ w n , ˆ z n ) w n March 07, 2018 | SCIP Workshop, Aachen | Frederic Matter | 9

Enforcing Modulus Constraints y ) of the LP relaxation violates w 2 n + z 2 If the solution ( ˆ w , ˆ z , ˆ n ≥ y n for some n ∈ [ N ]: ◮ If binary variable ˆ y n fixed to zero: set ˆ w n , ˆ z n to zero as well. ◮ If bounds of the continuous variables w n and z n are not yet restricted to one of the four orthants: create four branching nodes, one for each orthant. ◮ Otherwise: z n 1. Propagation, ( ˆ w n , ˆ z n ) 2. Separation, 3. Branching. w n March 07, 2018 | SCIP Workshop, Aachen | Frederic Matter | 9

Propagation z n If y n is fixed to one: ( l 1 , f ( l 1 )) l 1 , u 1 , l 2 , u 2 current lower and f ( l 1 ) ( f ( u 2 ), u 2 ) u ′ upper bounds of w n and z n . 2 = u 2 u 2 ( u 1 , f ( u 1 )) l ′ 1 = max { l 1 , f ( u 2 ) } , f ( u 1 ) u ′ 1 = min { u 1 , f ( l 2 ) } , l ′ 2 = f ( u 1 ) ( f ( l 2 ), l 2 ) l ′ 2 = max { l 2 , f ( u 1 ) } , l 2 u ′ 2 = min { u 2 , f ( l 1 ) } , w n √ u 1 l 1 f ( u 2 ) f ( l 2 ) 1 − x 2 . where f ( x ) = w 2 n + z 2 n ≥ y n needs to be l ′ u ′ 1 = f ( u 2 ) 1 = u 1 fulfilled. March 07, 2018 | SCIP Workshop, Aachen | Frederic Matter | 10

Propagation z n If y n is fixed to one: ( l 1 , f ( l 1 )) l 1 , u 1 , l 2 , u 2 current lower and f ( l 1 ) ( f ( u 2 ), u 2 ) u ′ upper bounds of w n and z n . 2 = u 2 u 2 ( u 1 , f ( u 1 )) l ′ 1 = max { l 1 , f ( u 2 ) } , f ( u 1 ) u ′ 1 = min { u 1 , f ( l 2 ) } , l ′ 2 = f ( u 1 ) ( f ( l 2 ), l 2 ) l ′ 2 = max { l 2 , f ( u 1 ) } , l 2 u ′ 2 = min { u 2 , f ( l 1 ) } , w n √ u 1 l 1 f ( u 2 ) f ( l 2 ) 1 − x 2 . where f ( x ) = w 2 n + z 2 n ≥ y n needs to be l ′ u ′ 1 = f ( u 2 ) 1 = u 1 fulfilled. ⇒ Use l ′ 1 , u ′ 1 , l ′ 2 and u ′ 2 as new lower and upper bounds of w n and z n . March 07, 2018 | SCIP Workshop, Aachen | Frederic Matter | 10

Separation and Branching If ˆ w n + ˆ z n < ˆ y n , add the cut w n + z n ≥ y n to the LP relaxation. z n w n March 07, 2018 | SCIP Workshop, Aachen | Frederic Matter | 11

Separation and Branching If ˆ w n + ˆ z n < ˆ y n , add the cut w n + z n ≥ y n to the LP relaxation. If ˆ w n + ˆ z n ≥ ˆ y n , create two branching nodes with the inequalities ′ − 1 ′ ′ − ˆ z n z n ˆ z n ˆ · w n + z n = 1, ′ − 1 · w n − z n = ′ − 1, ′ w n ˆ w n ˆ w n ˆ ′ = ′ = √ w n ˆ √ z n ˆ where ( ˆ w n , ˆ z n ) is the current LP solution, and ˆ w n z n 2 , ˆ z n z n 2 . w n 2 + ˆ w n 2 + ˆ ˆ ˆ z n z n z n w n w n w n March 07, 2018 | SCIP Workshop, Aachen | Frederic Matter | 11

Specialized Approach – Some remarks ◮ Enforcement of binary variables, second-order cone (SOC) constraint and the (convex) upper bound part of the quadratic modulus constraints is prioritized over the (nonconvex) lower bound part of the modulus constraints. ◮ A modulus constraint for enforcing is selected by a “most infeasible” rule: Enforce a modulus constraint w 2 n + z 2 n ≥ y ¯ n , ¯ n ∈ [ N ], with largest violation ¯ ¯ w 2 z 2 ρ ( n ) = ˆ y n − ( ˆ n + ˆ n ). March 07, 2018 | SCIP Workshop, Aachen | Frederic Matter | 12

Overview Motivation and Introduction Solving CMPs A heuristic Application Implementation Conclusion and Outline March 07, 2018 | SCIP Workshop, Aachen | Frederic Matter | 13