ENE 2XX: Renewable Energy Systems and Control LEC 07 : Convex - PowerPoint PPT Presentation

ENE 2XX: Renewable Energy Systems and Control LEC 07 : Convex Relaxations for Large-Scale MIQPs Professor Scott Moura University of California, Berkeley Summer 2018 Prof. Moura | Tsinghua-Berkeley Shenzhen Institute ENE 2XX | LEC 07 - Large-Scale MIQPs Slide 1

Collaborators eCAL Ph.D. Student @ UC Berkeley Bertrand Travacca Prof. Moura | Tsinghua-Berkeley Shenzhen Institute ENE 2XX | LEC 07 - Large-Scale MIQPs Slide 2

Problem Statement Consider the general mixed integer quadratically constrained quadratic program (MIQCQP): minimize f ( x ) (1) subject to: g j ( x ) ≤ 0 , j = 1 , · · · , m (2) 0 ≤ x ≤ 1 (3) x i ∈ { 0 , 1 } , i = 1 , · · · , p < n (4) x ∈ R n is the optimization variable the first p < n variables must be binary f ( · ) : R n → R is quadratic and L f – smooth g j ( · ) : R n → R are quadratic and L j – smooth Prof. Moura | Tsinghua-Berkeley Shenzhen Institute ENE 2XX | LEC 07 - Large-Scale MIQPs Slide 3

Problem Statement Consider the general mixed integer quadratically constrained quadratic program (MIQCQP): minimize f ( x ) (1) subject to: g j ( x ) ≤ 0 , j = 1 , · · · , m (2) 0 ≤ x ≤ 1 (3) x i ∈ { 0 , 1 } , i = 1 , · · · , p < n (4) x ∈ R n is the optimization variable the first p < n variables must be binary f ( · ) : R n → R is quadratic and L f – smooth g j ( · ) : R n → R are quadratic and L j – smooth Challenge Solve LARGE-SCALE MIQCQPs, e.g. n = 10 3 , 10 4 , 10 5 , · · · Prof. Moura | Tsinghua-Berkeley Shenzhen Institute ENE 2XX | LEC 07 - Large-Scale MIQPs Slide 3

Problem Statement Consider the general mixed integer quadratically constrained quadratic program (MIQCQP): minimize f ( x ) (1) subject to: g j ( x ) ≤ 0 , j = 1 , · · · , m (2) 0 ≤ x ≤ 1 (3) x i ∈ { 0 , 1 } , i = 1 , · · · , p < n (4) x ∈ R n is the optimization variable the first p < n variables must be binary f ( · ) : R n → R is quadratic and L f – smooth g j ( · ) : R n → R are quadratic and L j – smooth Challenge Solve LARGE-SCALE MIQCQPs, e.g. n = 10 3 , 10 4 , 10 5 , · · · P vs NP – Millenium Prize Problem Prof. Moura | Tsinghua-Berkeley Shenzhen Institute ENE 2XX | LEC 07 - Large-Scale MIQPs Slide 3

Outline Existing Convex Relaxation Methods 1 Hopfield Methods 2 Dual Hopfield Methods 3 Simulations: Solving random MIQPs 4 Summary 5 Prof. Moura | Tsinghua-Berkeley Shenzhen Institute ENE 2XX | LEC 07 - Large-Scale MIQPs Slide 4

Existing Methods Meta-heuristics Meta-heuristics for mixed-integer problems simulated annealing tabu search genetic algorithm particle swarm optimization Prof. Moura | Tsinghua-Berkeley Shenzhen Institute ENE 2XX | LEC 07 - Large-Scale MIQPs Slide 5

Existing Methods Meta-heuristics Meta-heuristics for mixed-integer problems simulated annealing tabu search genetic algorithm particle swarm optimization Advantages Disadvantages + “Black-box” – Does not exploit structure + Open-source codes exists – Convergence results don’t exist, in general – Does not scale well, in general Prof. Moura | Tsinghua-Berkeley Shenzhen Institute ENE 2XX | LEC 07 - Large-Scale MIQPs Slide 5

Existing Methods Convex Relaxation #1: Binary Relaxation If f ( x ) , g j ( x ) are convex, then relax binary constraints to x i ∈ [ 0 , 1 ] ... f ( x ) minimize (5) g j ( x ) ≤ 0 , j = 1 , · · · , m subject to: (6) 0 ≤ x ≤ 1 (7) x i ∈ { 0 , 1 } , i = 1 , · · · , p < n (8) then use interior-point methods Prof. Moura | Tsinghua-Berkeley Shenzhen Institute ENE 2XX | LEC 07 - Large-Scale MIQPs Slide 6

Existing Methods Convex Relaxation #1: Binary Relaxation Stochastic approach to recover integer constraint: Let x r be solution to binary relaxation. Feasible x can be drawn randomly from { 0 , 1 } following Bernoulli distribution B ( x r ) . This can be sub-optimal. Prof. Moura | Tsinghua-Berkeley Shenzhen Institute ENE 2XX | LEC 07 - Large-Scale MIQPs Slide 7

Existing Methods Convex Relaxation #1: Binary Relaxation Stochastic approach to recover integer constraint: Let x r be solution to binary relaxation. Feasible x can be drawn randomly from { 0 , 1 } following Bernoulli distribution B ( x r ) . This can be sub-optimal. Example � 2 � x − 1 = 1 ( x ⋆ = 0 is the optimal solution ) minimize x ∈{ 0 , 1 } 4 16 � 2 = If we apply binary relaxation, we get x r = 1 x − 1 3 1 � 4 and E x ∼B ( x r ) 16 > 16 ! 4 Other ideas: Branch & Bound, Branch & Cut Prof. Moura | Tsinghua-Berkeley Shenzhen Institute ENE 2XX | LEC 07 - Large-Scale MIQPs Slide 7

Existing Methods Convex Relaxation #2: Lagrangian Relaxation Notice that x i ∈ { 0 , 1 } is equivalent to satisfying x i ( 1 − x i ) = 0 minimize f ( x ) (9) subject to: g j ( x ) ≤ 0 , j = 1 , · · · , m (10) 0 ≤ x ≤ 1 (11) x i ( 1 − x i ) = 0 , i = 1 , · · · , p < n (12) Prof. Moura | Tsinghua-Berkeley Shenzhen Institute ENE 2XX | LEC 07 - Large-Scale MIQPs Slide 8

Existing Methods Convex Relaxation #2: Lagrangian Relaxation Notice that x i ∈ { 0 , 1 } is equivalent to satisfying x i ( 1 − x i ) = 0 minimize f ( x ) (9) subject to: g j ( x ) ≤ 0 , j = 1 , · · · , m (10) 0 ≤ x ≤ 1 (11) x i ( 1 − x i ) = 0 , i = 1 , · · · , p < n (12) Form the Lagrangian : p m � � � � L ( x , µ, µ, µ, λ ) = f ( x ) + µ j g j ( x ) + µ j x i + µ j ( 1 − x i ) + λ i x i ( 1 − x i ) (13) j = 1 i = 1 Prof. Moura | Tsinghua-Berkeley Shenzhen Institute ENE 2XX | LEC 07 - Large-Scale MIQPs Slide 8

Existing Methods Convex Relaxation #2: Lagrangian Relaxation Notice that x i ∈ { 0 , 1 } is equivalent to satisfying x i ( 1 − x i ) = 0 minimize f ( x ) (9) subject to: g j ( x ) ≤ 0 , j = 1 , · · · , m (10) 0 ≤ x ≤ 1 (11) x i ( 1 − x i ) = 0 , i = 1 , · · · , p < n (12) Form the Lagrangian : p m � � � � L ( x , µ, µ, µ, λ ) = f ( x ) + µ j g j ( x ) + µ j x i + µ j ( 1 − x i ) + λ i x i ( 1 − x i ) (13) j = 1 i = 1 Define the (concave) dual function of Λ = [ µ, µ, µ, λ ] D (Λ) = min x ∈ R n L ( x , µ, µ, µ, λ ) (14) Weak duality approach: Solve convex program max Λ D (Λ) Prof. Moura | Tsinghua-Berkeley Shenzhen Institute ENE 2XX | LEC 07 - Large-Scale MIQPs Slide 8

Existing Methods Convex Relaxation #3: Semi-definite Relaxation Introduce new variable X = xx T . This is called “lifting”. Can re-write MIQCQP Prof. Moura | Tsinghua-Berkeley Shenzhen Institute ENE 2XX | LEC 07 - Large-Scale MIQPs Slide 9

Existing Methods Convex Relaxation #3: Semi-definite Relaxation Introduce new variable X = xx T . This is called “lifting”. Can re-write MIQCQP 1 2 Tr ( QX ) + R T x + S minimize (15) 1 2 Tr ( Q j X ) + R T subject to: j x + S j ≤ 0 , j = 1 , · · · , m (16) 0 ≤ x ≤ 1 (17) X ii = x i , i = 1 , · · · , p < n (18) X = xx T (19) If Q , Q i are positive semi-definite, then only X = xx T makes this non-convex. Prof. Moura | Tsinghua-Berkeley Shenzhen Institute ENE 2XX | LEC 07 - Large-Scale MIQPs Slide 9

Existing Methods Convex Relaxation #3: Semi-definite Relaxation Introduce new variable X = xx T . This is called “lifting”. Can re-write MIQCQP 1 2 Tr ( QX ) + R T x + S minimize (15) 1 2 Tr ( Q j X ) + R T subject to: j x + S j ≤ 0 , j = 1 , · · · , m (16) 0 ≤ x ≤ 1 (17) X ii = x i , i = 1 , · · · , p < n (18) X = xx T (19) If Q , Q i are positive semi-definite, then only X = xx T makes this non-convex. Relax into convex inequality X � xx T . Using Schur complement: � � X x X � xx T ⇔ � 0 (20) x 1 This can be cast as a semi-definite program (SDP). Prof. Moura | Tsinghua-Berkeley Shenzhen Institute ENE 2XX | LEC 07 - Large-Scale MIQPs Slide 9

Outline Existing Convex Relaxation Methods 1 Hopfield Methods 2 Dual Hopfield Methods 3 Simulations: Solving random MIQPs 4 Summary 5 Prof. Moura | Tsinghua-Berkeley Shenzhen Institute ENE 2XX | LEC 07 - Large-Scale MIQPs Slide 10

A short history In 1982, J. J. Hopfield used neural nets to model agents that do collaborative computations In 1985, J. J. Hopfield showed that neural nets can be used to solve optimization problems In 1990’s, Hopfield methods became very popular for solving MIQPs in power systems optimization In literature, power system engineers admit they didn’t fully understand why Hopfield methods work well. Prof. Moura | Tsinghua-Berkeley Shenzhen Institute ENE 2XX | LEC 07 - Large-Scale MIQPs Slide 11

The Hopfield Method Consider MIQP f ( x ) = 1 2 x T Qx minimize (21) subject to: 0 ≤ x i ≤ 1 i = 1 , · · · , n (22) x i ∈ { 0 , 1 } i = 1 , · · · , p < n (23) Prof. Moura | Tsinghua-Berkeley Shenzhen Institute ENE 2XX | LEC 07 - Large-Scale MIQPs Slide 12

The Hopfield Method Consider MIQP f ( x ) = 1 2 x T Qx minimize (21) subject to: 0 ≤ x i ≤ 1 i = 1 , · · · , n (22) x i ∈ { 0 , 1 } i = 1 , · · · , p < n (23) The Hopfield method follows the dynamical system: d x H ( 0 ) = x ( 0 ) ∈ ( 0 , 1 ) n dt x H ( t ) = − Qx ( t ); (24) x ( t ) = σ ( x H ( t )) (25) where σ ( · ) : R n → [ 0 , 1 ] is an “activiation function” defined element-wise as: σ ( x ) : x �→ [ σ 1 ( x 1 ) , · · · , σ n ( x n )] Prof. Moura | Tsinghua-Berkeley Shenzhen Institute ENE 2XX | LEC 07 - Large-Scale MIQPs Slide 12