Back to the Roots Polynomial System Solving Using Linear Algebra - PowerPoint PPT Presentation

Motivation and History Univariate Polynomials Multivariate Polynomials Algebraic Optimization Conclusions Back to the Roots Polynomial System Solving Using Linear Algebra Philippe Dreesen KU Leuven Department of Electrical Engineering ESAT-STADIUS Stadius Center for Dynamical Systems, Signal Processing and Data Analytics 1 / 42

Motivation and History Univariate Polynomials Multivariate Polynomials Algebraic Optimization Conclusions Outline Motivation and History 1 Univariate Polynomials 2 Multivariate Polynomials 3 Algebraic Optimization 4 Conclusions 5 2 / 42

Motivation and History Univariate Polynomials Multivariate Polynomials Algebraic Optimization Conclusions Four instances of polynomial root-finding problems � x − 1 � ( x − 1)( x − 3)( x − 2) = 0 ( x − 1)( x + 2) = 0 3 − ( x − 2)( x − 3) = 0 x 2 + y 2 min x 2 + 3 y 2 − 15 = 0 x,y y − 3 x 3 − 2 x 2 + 13 x − 2 y − x 2 + 2 x − 1 = 0 = 0 s . t . 3 / 42

Motivation and History Univariate Polynomials Multivariate Polynomials Algebraic Optimization Conclusions Why polynomials? Why Study Polynomial Equations? – fundamental mathematical objects – powerful modelling tools – ubiquitous in Science and Engineering (often hidden ) Systems and Control Signal Processing Computational Biology Kinematics/Robotics 4 / 42

Motivation and History Univariate Polynomials Multivariate Polynomials Algebraic Optimization Conclusions A long and rich history. . . Egypt Babylon Euclid Diophantus Al-Khwarizmi (3000BCE-300BCE) (3000BCE-539BCE) (fl. 300BCE) (c200-c284) (c780-c850) Zhu Shijie Pierre de Fermat Ren´ e Descartes Isaac Newton Gottfried Leibniz (c1260-c1320) (c1601-1665) (1596-1650) (1643-1727) (1646-1716) 5 / 42

Motivation and History Univariate Polynomials Multivariate Polynomials Algebraic Optimization Conclusions . . . leading to “Algebraic Geometry” Etienne B´ ezout Carl Friedrich Gauss Jean-Victor Poncelet Evariste Galois Arthur Cayley (1730-1783) (1777-1755) (1788-1867) (1811-1832) (1821-1895) Leopold Kronecker Edmond Laguerre James J. Sylvester Francis S. Macaulay David Hilbert (1823-1891) (1834-1886) (1814-1897) (1862-1937) (1862-1943) 6 / 42

Motivation and History Univariate Polynomials Multivariate Polynomials Algebraic Optimization Conclusions . . . leading to “Algebraic Geometry” Algebraic Geometry and Computer Algebra – large body of literature – emphasis not (anymore) on solving equations – computer algebra: symbolic manipulations (e.g., Gr¨ obner Bases) – numerical issues! Wolfgang Gr¨ obner Bruno Buchberger (1899-1980) 7 / 42

Motivation and History Univariate Polynomials Multivariate Polynomials Algebraic Optimization Conclusions . . . and (Numerical) Linear Algebra Joseph-Louis Lagrange Augustin-Louis Cauchy Hermann Grassmann Charles Babbage Ada Lovelace (1736-1813) (1789-1857) (1809-1877) (1791-1871) (1815-1852) Alan Turing John von Neumann Gene Golub Daniel Lazard Hans J. Stetter (1912-1954) (1903-1957) (1932-2007) 8 / 42

Motivation and History Univariate Polynomials Multivariate Polynomials Algebraic Optimization Conclusions . . . and (Numerical) Linear Algebra Why Linear Algebra? – comprehensible and accessible language – intuitive geometric interpretation – computationally powerful framework – well-established methods and stable numerics 9 / 42

Motivation and History Univariate Polynomials Multivariate Polynomials Algebraic Optimization Conclusions . . . and (Numerical) Linear Algebra Eigenvalue Problems Eigenvalue equation Av = λv and eigenvalue decomposition A = V Λ V − 1 Enormous importance in (numerical) linear algebra and apps – ‘understand’ the action of matrix A – at the heart of a multitude of applications: oscillations, vibrations, quantum mechanics, data analytics, graph theory, and many more 10 / 42

Motivation and History Univariate Polynomials Multivariate Polynomials Algebraic Optimization Conclusions Outline Motivation and History 1 Univariate Polynomials 2 Multivariate Polynomials 3 Algebraic Optimization 4 Conclusions 5 12 / 42

Motivation and History Univariate Polynomials Multivariate Polynomials Algebraic Optimization Conclusions Well-known facts Univariate Polynomials and Linear Algebra: Known Facts Characteristic Polynomial The eigenvalues of A are the roots of p ( λ ) = | A − λI | Companion Matrix Solving q ( x ) = 7 x 3 − 2 x 2 − 5 x + 1 = 0 leads to       0 1 0 1 1  = x 0 0 1 x x      x 2 x 2 − 1 / 7 5 / 7 2 / 7 13 / 42

Motivation and History Univariate Polynomials Multivariate Polynomials Algebraic Optimization Conclusions A less well-known method Sylvester Matrix Consider two polynomial equations x 3 − 6 x 2 + 11 x − 6 f ( x ) = = ( x − 1)( x − 2)( x − 3) − x 2 + 5 x − 6 g ( x ) = = − ( x − 2)( x − 3) Common roots if | S ( f, g ) | = 0  − 6 11 − 6 1 0  0 − 6 11 − 6 1     S ( f, g ) = − 6 5 − 1 0 0     0 − 6 5 − 1 0 James Joseph Sylvester   0 0 − 6 5 − 1 14 / 42

Motivation and History Univariate Polynomials Multivariate Polynomials Algebraic Optimization Conclusions A less well-known method Sylvester’s construction can be understood from x 2 x 3 x 4 1 x  − 6 11 − 6 1 0   1 1  f ( x )=0 − 6 11 − 6 1 x 1 x 2 x · f ( x )=0        x 2 x 2  − 6 5 − 1 = 0 g ( x )=0    1 2     x 3 x 3  − 6 5 − 1 x · g ( x )=0    1 2  x 4 x 4 − 6 5 − 1 x 2 · g ( x )=0 1 2 where x 1 = 2 and x 2 = 3 are the common roots of f and g 15 / 42

Motivation and History Univariate Polynomials Multivariate Polynomials Algebraic Optimization Conclusions Outline Motivation and History 1 Univariate Polynomials 2 Multivariate Polynomials 3 Algebraic Optimization 4 Conclusions 5 16 / 42

Motivation and History Univariate Polynomials Multivariate Polynomials Algebraic Optimization Conclusions Null space based Root-finding Consider the system x 2 + 3 y 2 − 15 p ( x, y ) = = 0 y − 3 x 3 − 2 x 2 + 13 x − 2 q ( x, y ) = = 0 Matrix representation of the system: Macaulay matrix M x 2 y 2 x 3 x 2 y xy 2 y 3 1 x y xy   − 15 1 3 p ( x,y ) − 15 1 3 x · p ( x,y )     − 15 1 3 y · p ( x,y )   − 2 13 1 − 2 − 3 q ( x,y ) 17 / 42

Motivation and History Univariate Polynomials Multivariate Polynomials Algebraic Optimization Conclusions Null space based Root-finding x 2 + 3 y 2 − 15 p ( x, y ) = = 0 y − 3 x 3 − 2 x 2 + 13 x − 2 q ( x, y ) = = 0 Continue to enlarge the Macaulay matrix M : x 5 x 4 yx 3 y 2 x 2 y 3 xy 4 y 5 → x 2 y 2 x 3 x 2 y xy 2 y 3 x 4 x 3 yx 2 y 2 xy 3 y 4 1 x y xy p − 15 1 3 xp − 15 1 3 d = 3 yp − 15 1 3 q − 2 13 1 − 2 − 3 x 2 p − 15 1 3 xyp − 15 1 3 y 2 p d = 4 − 15 1 3 xq − 2 13 1 − 2 − 3 yq − 2 13 1 − 2 − 3 x 3 p − 15 1 3 x 2 yp − 15 1 3 xy 2 p − 15 1 3 d = 5 y 3 p − 15 1 3 x 2 q − 2 13 1 − 2 − 3 xyq − 2 13 1 − 2 − 3 y 2 q − 2 13 1 − 2 − 3 ↓ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 / 42

Motivation and History Univariate Polynomials Multivariate Polynomials Algebraic Optimization Conclusions Null space based Root-finding Multivariate Vandermonde – Macaulay coefficient matrix M : basis for the null space: � × 0 0 0 � × × ×   1 1 . . . 1 0 0 0 M = × × × × 0 0 0 × × × × x 1 x 2 . . . x m   0 0 0 × × × ×   y 1 y 2 . . . y m     – solutions generate vectors in null space x 2 x 2 x 2 . . .   1 2 m    x 1 y 1 x 2 y 2 . . . x m y m  MK = 0    y 2 y 2 y 2  . . . 1 2 m    x 3 x 3 x 3  – number of solutions m = nullity . . . m  1 2    x 2 x 2 x 2 1 y 1 2 y 2 . . . m y m     x 1 y 2 x 2 y 2 x m y 2 . . .   m 1 2   y 3 y 3 y 3 . . .   1 2 m   x 4 x 4 x 4 . . .   1 2 4   x 3 x 3 x 3   1 y 1 2 y 2 . . . m y m    x 2 1 y 2 x 2 2 y 2 x 2 m y 2  . . .  1 2 m   x 1 y 3 x 2 y 3 x m y 3  . . .  1 2 m   y 4 y 4 y 4  . . .  m  1 2   . . . . . . . . . . . . 19 / 42