Foreword IPMs Linear Algebra in IPMs Tulip.jl Conclusion References Tulip.jl : an interior-point solver with abstract linear algebra Miguel Anjos a , b Andrea Lodi a , b , c Mathieu Tanneau a , b , c (a) École polytechnique de Montréal (b) GERAD (c) CERC in Data Science for Real-time decision making March 13, 2019 Mathieu Tanneau Tulip.jl - March 13, 2019 1 / 24
Foreword IPMs Linear Algebra in IPMs Tulip.jl Conclusion References 1 Foreword Interior-Point Methods 2 Linear Algebra in IPMs 3 Tulip.jl 4 Conclusion 5 Mathieu Tanneau Tulip.jl - March 13, 2019 2 / 24
Foreword IPMs Linear Algebra in IPMs Tulip.jl Conclusion References Linear programming (Primal-Dual standard form) c T x b T y ( P ) min ( D ) max y , s x A T y s . t . A x = b + s = c s . t . x ≥ 0 ≥ 0 s where x ∈ R n , y ∈ R m , s ∈ R n Solved with Simplex or Interior-Point Workhorse of MILP Decomposition (Dantzig-Wolfe & Benders) Polyhedral Outer approximations Cutting plane methods ... Mathieu Tanneau Tulip.jl - March 13, 2019 3 / 24
Foreword IPMs Linear Algebra in IPMs Tulip.jl Conclusion References Geometric view c T x ( P ) min x min s . t . A x = b x ≥ 0 Simplex - Many cheap iterations - Extreme vertices (basic points) Interior-Point - Few expensive iterations - Interior points ( x > 0) Mathieu Tanneau Tulip.jl - March 13, 2019 4 / 24
Foreword IPMs Linear Algebra in IPMs Tulip.jl Conclusion References 1 Foreword Interior-Point Methods 2 Linear Algebra in IPMs 3 Tulip.jl 4 Conclusion 5 Mathieu Tanneau Tulip.jl - March 13, 2019 5 / 24
Foreword IPMs Linear Algebra in IPMs Tulip.jl Conclusion References IPM overview c T x b T y ( D ) max ( P ) min y , s x A x = b A T y s . t . s . t . + s = c x ≥ 0 s ≥ 0 KKT optimality conditions: A x = b [primal feas.] (1) A T y + s = c [dual feas.] (2) ∀ i , x i · s i = 0 [slackness] (3) x , s ≥ 0 (4) Algorithm (1) (2) (3) x ≥ 0 s ≥ 0 (primal) Simplex * � � � � Interior-point * x > 0 s > 0 � � � : at each iteration; *: at optimality only Mathieu Tanneau Tulip.jl - March 13, 2019 6 / 24
Foreword IPMs Linear Algebra in IPMs Tulip.jl Conclusion References IPM overview Short history of IPMs: The seminal paper [Karmarkar, 1984] [Mehrotra, 1992]: predictor-corrector algorithm (implemented in most IPM codes) Multiple centrality corrections [Gondzio, 1996] Reference textbook [Wright, 1997] [Gondzio, 2012]: more recent survey of IPMs (Some) software for LP/QP: All commercial solvers (CPLEX, GRB, Mosek, Xpress, etc.) Open source: CLP, GLPK, OOQP, (PCx), (HOPDM) Mathieu Tanneau Tulip.jl - March 13, 2019 7 / 24
Foreword IPMs Linear Algebra in IPMs Tulip.jl Conclusion References Mehrotra’s Predictor-Corrector algorithm Compute initial point (see [Mehrotra, 1992]) ( x 0 , y 0 , s 0 ) with x 0 > 0 , s 0 > 0 Compute search direction ∆ x aff A 0 0 b − A x · = A T ∆ y aff c − A T y − s 0 I [predictor] ∆ s aff S 0 X − XSe ∆ x cc A 0 0 0 · = A T ∆ y cc 0 I 0 [corrector] σµ e − ∆ X aff ∆ S aff ∆ s cc S 0 X Update current solution ( x + , y + , s + ) = ( x , y , s ) + α (∆ aff + ∆ cc ) Repeat until convergence Mathieu Tanneau Tulip.jl - March 13, 2019 8 / 24
Foreword IPMs Linear Algebra in IPMs Tulip.jl Conclusion References Mehrotra’s Predictor-Corrector algorithm min Affine-scaling (full) Affine-scaling (damped) Corrector Predictor-corrector step Figure: Mehrotra’s Predictor-Corrector, in x space Mathieu Tanneau Tulip.jl - March 13, 2019 9 / 24
Foreword IPMs Linear Algebra in IPMs Tulip.jl Conclusion References Wrap-up LP in standard Primal-Dual form b T y c T x ( P ) min ( D ) max y , s x A T y s . t . A x = b + s = c s . t . x ≥ 0 ≥ 0 s At each iteration, solve (several) Newton systems of the form A T 0 I ∆ x ξ d = · A 0 0 ∆ y ξ p S 0 X ∆ s ξ xs Polynomial-time algorithm (see [Wright, 1997]) Very efficient on large-scale problems Mathieu Tanneau Tulip.jl - March 13, 2019 10 / 24
Foreword IPMs Linear Algebra in IPMs Tulip.jl Conclusion References 1 Foreword Interior-Point Methods 2 Linear Algebra in IPMs 3 Tulip.jl 4 Conclusion 5 Mathieu Tanneau Tulip.jl - March 13, 2019 11 / 24
Foreword IPMs Linear Algebra in IPMs Tulip.jl Conclusion References Newton system Newton systems of the form A T 0 I ∆ x ξ d = · A 0 0 ∆ y ξ p S 0 X ∆ s ξ xs solved multiple times in each iteration, with various right-hand side. Two ways to make an Interior-Point faster: Reduce the number of iterations (better algorithm) Reduce the time per iteration (better linear algebra) Mathieu Tanneau Tulip.jl - March 13, 2019 12 / 24
Foreword IPMs Linear Algebra in IPMs Tulip.jl Conclusion References Augmented system Initial Newton system: A T 0 I ∆ x ξ d · = A 0 0 ∆ y ξ p S 0 X ∆ s ξ xs Substitute ∆ s to obtain the Augmented system � ∆ x � ξ d − X − 1 ξ xs − Θ − 1 � A T � � � = · 0 ∆ y A ξ p ∆ s = X − 1 ( ξ xs − S ∆ x ) where Θ := XS − 1 :( Left-hand matrix is indefinite (though regularization can be used) :( Still costly to solve :) More handy if free variables and/or non-linear terms Mathieu Tanneau Tulip.jl - March 13, 2019 13 / 24
Foreword IPMs Linear Algebra in IPMs Tulip.jl Conclusion References Normal equations Substitute ∆ x to obtain the Normal equations ( A Θ A T )∆ y = ξ p + A Θ( ξ d − X − 1 ξ xs ) ∆ x = Θ( A T ∆ y − ξ d + X − 1 ξ xs ) ∆ s = X − 1 ( ξ xs − S ∆ x ) :) A Θ A T is positive-definite :) Cholesky factorization A Θ A T = LL T = ⇒ specialized Cholesky based on A Mathieu Tanneau Tulip.jl - March 13, 2019 14 / 24
Foreword IPMs Linear Algebra in IPMs Tulip.jl Conclusion References Specialized Cholesky Unit block-angular matrix e T ... A = e T A 1 · · · A N Found in Dantzig-Wolfe decomposition + column-generation e T θ 1 ( A 1 θ 1 ) T . ... . A Θ A T = . e T θ R ( A R θ R ) T A 1 θ 1 A R θ R Φ · · · = ⇒ exploit structure to accelerate Cholesky factorization Mathieu Tanneau Tulip.jl - March 13, 2019 15 / 24
Foreword IPMs Linear Algebra in IPMs Tulip.jl Conclusion References 1 Foreword Interior-Point Methods 2 Linear Algebra in IPMs 3 Tulip.jl 4 Conclusion 5 Mathieu Tanneau Tulip.jl - March 13, 2019 16 / 24
Foreword IPMs Linear Algebra in IPMs Tulip.jl Conclusion References Solver overview https://github.com/ds4dm/Tulip.jl Main features Homogeneous self-dual algorithm + multiple corrections Upper-bounds handled explicitly Algorithm uses abstract linear algebra ( A::AbstractMatrix ) Generic sparse Cholesky + specialized for Unit block-angular MathProgBase interface WIP MOI interface Improved stability & general sparse linear algebra Mathieu Tanneau Tulip.jl - March 13, 2019 17 / 24
Foreword IPMs Linear Algebra in IPMs Tulip.jl Conclusion References Netlib instances Netlib benchmark ( https://www.netlib.org/lp/ ) Small LP instances, some problematic Only consider feasible instances with no free variables No presolve, no crossover, single thread Most solved in < 1s Results Tulip runs into numerical issues numerical issues, but... ...faster than CLP, GLPK, IpOpt on “hard" instances (hard = solved in > 0 . 1s by all solvers) Mathieu Tanneau Tulip.jl - March 13, 2019 18 / 24
Foreword IPMs Linear Algebra in IPMs Tulip.jl Conclusion References Column-generation instances Instances: m = 24 , 48 , 96 linking constraints N = 2 10 to 2 15 sub-problems Each sub-problem solved with Gurobi Same column-generation code Master problem statistics: N + m constraints up to ≃ 4 − 10 × N variables ≃ 4 − 10 × N × m non-zeros Mathieu Tanneau Tulip.jl - March 13, 2019 19 / 24
Foreword IPMs Linear Algebra in IPMs Tulip.jl Conclusion References Column-generation instances Solver setup: Barrier algorithm, no cross-over No presolve Single thread Tulip: Generic IPM + specialized linear algebra Computational results: Barrier (almost always) faster than Simplex Computing times (for Restricted Master Problem) vs Mosek: − 33% (total time); − 50% (per-iteration time) vs Gurobi: − 60% (total time); − 70% (per-iteration time) vs CPLEX: − 55% (total time); − 45% (per-iteration time) Mathieu Tanneau Tulip.jl - March 13, 2019 20 / 24
Foreword IPMs Linear Algebra in IPMs Tulip.jl Conclusion References 1 Foreword Interior-Point Methods 2 Linear Algebra in IPMs 3 Tulip.jl 4 Conclusion 5 Mathieu Tanneau Tulip.jl - March 13, 2019 21 / 24
Foreword IPMs Linear Algebra in IPMs Tulip.jl Conclusion References Conclusion Takeaway: IPM solver for linear programming Generic algorithm + specialized linear algebra Possible to beat SOTA solvers Roadmap: MOI interface Numerical stability Extension to QP Open JuMP-related questions: Passing structure information to solver Problem modification Mathieu Tanneau Tulip.jl - March 13, 2019 22 / 24
Recommend
More recommend
