A Users Guide to Riemannian Newton-Type Methods on Manifolds Felipe - PowerPoint PPT Presentation

Motivation Outline A User’s Guide to Riemannian Newton-Type Methods on Manifolds Felipe Álvarez Departamento de Ingeniería Matemática Centro de Modelamiento Matemático (CNRS UMI 2807) Universidad de Chile In collaboration with: J. Bolte, J. Munier, J. López Sixièmes Journées Franco-Chiliennes d’Optimisation Université du Sud Toulon-Var Mai 19-21, 2008 http://www.dim.uchile.cl/~falvarez Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 1 / 23

Motivation Outline Motivation: Nonlinear equations in a manifold Goal: find p ∗ ∈ M satisfying F ( p ∗ ) = 0 ∈ T p ∗ M M is a connected and n -dimensional differentiable manifold. T p M ≃ R n is the tangent space of M at p : If c ( t ) is a curve passing through p at t = 0 then ˙ c ( 0 ) ∈ T p M . F : M → TM is a continuously differentiable vector field: M ∋ p �→ F ( p ) ∈ T p M Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 2 / 23

Motivation Outline Motivation: Nonlinear equations in a manifold Goal: find p ∗ ∈ M satisfying F ( p ∗ ) = 0 ∈ T p ∗ M M is a connected and n -dimensional differentiable manifold. T p M ≃ R n is the tangent space of M at p : If c ( t ) is a curve passing through p at t = 0 then ˙ c ( 0 ) ∈ T p M . F : M → TM is a continuously differentiable vector field: M ∋ p �→ F ( p ) ∈ T p M Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 2 / 23

Motivation Outline Motivation: Nonlinear equations in a manifold Goal: find p ∗ ∈ M satisfying F ( p ∗ ) = 0 ∈ T p ∗ M M is a connected and n -dimensional differentiable manifold. T p M ≃ R n is the tangent space of M at p : If c ( t ) is a curve passing through p at t = 0 then ˙ c ( 0 ) ∈ T p M . F : M → TM is a continuously differentiable vector field: M ∋ p �→ F ( p ) ∈ T p M Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 2 / 23

Motivation Outline Motivation: Nonlinear equations in a manifold Goal: find p ∗ ∈ M satisfying F ( p ∗ ) = 0 ∈ T p ∗ M M is a connected and n -dimensional differentiable manifold. T p M ≃ R n is the tangent space of M at p : If c ( t ) is a curve passing through p at t = 0 then ˙ c ( 0 ) ∈ T p M . F : M → TM is a continuously differentiable vector field: M ∋ p �→ F ( p ) ∈ T p M Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 2 / 23

Motivation Outline Example 1: Rayleigh’s quotient on the sphere M = S n = { x ∈ R n + 1 | x T x = 1 } (unit sphere in R n + 1 ). T x M = { v ∈ R n + 1 | x T v = 0 } . F ( x ) = Ax − q ( x ) x with � A ∈ R n × n being symmetric and positive definite. � q ( x ) = x T Ax . x T F ( x ) = 0 ⇒ F ( x ) ∈ T x M . F ( x ∗ ) = 0 iff x ∗ is an eigenvector of A with q ( x ∗ ) the corresponding eigenvalue. Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 3 / 23

Motivation Outline Example 2: Stiefel manifold M = S n , k = { Y ∈ R n × k | Y T Y = I k } . T Y S n , k = { ∆ ∈ R n × k | ∆ T Y + Y T ∆ = 0 } . If k = 1 then S n , 1 = S n − 1 . If k = n then S n , n = O n the orthogonal group. T I n O n = { ∆ ∈ R n × n | ∆ T = − ∆ } . dim S n , k = nk − 1 2 k ( k + 1 ) . F ( Y ) = AY − YY T AY F ( Y ∗ ) = 0 iff the columns of Y ∗ are eigenvectors of A . Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 4 / 23

Motivation Outline Example 2: Stiefel manifold M = S n , k = { Y ∈ R n × k | Y T Y = I k } . T Y S n , k = { ∆ ∈ R n × k | ∆ T Y + Y T ∆ = 0 } . If k = 1 then S n , 1 = S n − 1 . If k = n then S n , n = O n the orthogonal group. T I n O n = { ∆ ∈ R n × n | ∆ T = − ∆ } . dim S n , k = nk − 1 2 k ( k + 1 ) . F ( Y ) = AY − YY T AY F ( Y ∗ ) = 0 iff the columns of Y ∗ are eigenvectors of A . Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 4 / 23

Motivation Outline Example 2: Stiefel manifold M = S n , k = { Y ∈ R n × k | Y T Y = I k } . T Y S n , k = { ∆ ∈ R n × k | ∆ T Y + Y T ∆ = 0 } . If k = 1 then S n , 1 = S n − 1 . If k = n then S n , n = O n the orthogonal group. T I n O n = { ∆ ∈ R n × n | ∆ T = − ∆ } . dim S n , k = nk − 1 2 k ( k + 1 ) . F ( Y ) = AY − YY T AY F ( Y ∗ ) = 0 iff the columns of Y ∗ are eigenvectors of A . Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 4 / 23

Motivation Outline Example 2: Stiefel manifold M = S n , k = { Y ∈ R n × k | Y T Y = I k } . T Y S n , k = { ∆ ∈ R n × k | ∆ T Y + Y T ∆ = 0 } . If k = 1 then S n , 1 = S n − 1 . If k = n then S n , n = O n the orthogonal group. T I n O n = { ∆ ∈ R n × n | ∆ T = − ∆ } . dim S n , k = nk − 1 2 k ( k + 1 ) . F ( Y ) = AY − YY T AY F ( Y ∗ ) = 0 iff the columns of Y ∗ are eigenvectors of A . Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 4 / 23

Motivation Outline Solving nonlinear equations: Euclidean case Goal: find p ∗ ∈ Ω such that F ( p ∗ ) = 0 ∈ R n , where Ω is open and F : Ω ⊂ R n → R n is a C 1 vector field. Newton’s method: F ( p k ) + F ′ ( p k )( p k + 1 − p k ) = 0 . 10 8 6 4 2 0 p k −2 p k+1 p* −4 −1 −0.5 0 0.5 1 1.5 2 2.5 Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 5 / 23

Motivation Outline Outline Abstract differential geometry setting for R-Newton 1 2 Other explicit examples Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 6 / 23

Abstract differential geometry setting for R-Newton Other explicit examples Outline Abstract differential geometry setting for R-Newton 1 2 Other explicit examples Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 7 / 23

Abstract differential geometry setting for R-Newton Other explicit examples Metric framework M is endowed with a Riemannian metric g : � v � 2 p = g ( p )( v , v ) for v ∈ T p M . Riemannian distance d : M × M → [ 0 , + ∞ ) : � b d ( p , q ) = inf { a � ˙ c ( t ) � c ( t ) dt | c : [ a , b ] → M , c ( a ) = p , c ( b ) = q } Assumption: ( M , d ) is a complete metric space. Covariant derivative: F ′ ( p ) v := ∇ v F ( p ) = ( ∇ Y F )( p ) , v ∈ T p M , where � Y is any vector field on M satisfying v = Y ( p ) . � ∇ is the Riemannian (or Levi-Civita) connection on ( M , g ) . Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 8 / 23

Abstract differential geometry setting for R-Newton Other explicit examples Metric framework M is endowed with a Riemannian metric g : � v � 2 p = g ( p )( v , v ) for v ∈ T p M . Riemannian distance d : M × M → [ 0 , + ∞ ) : � b d ( p , q ) = inf { a � ˙ c ( t ) � c ( t ) dt | c : [ a , b ] → M , c ( a ) = p , c ( b ) = q } Assumption: ( M , d ) is a complete metric space. Covariant derivative: F ′ ( p ) v := ∇ v F ( p ) = ( ∇ Y F )( p ) , v ∈ T p M , where � Y is any vector field on M satisfying v = Y ( p ) . � ∇ is the Riemannian (or Levi-Civita) connection on ( M , g ) . Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 8 / 23

Abstract differential geometry setting for R-Newton Other explicit examples Metric framework M is endowed with a Riemannian metric g : � v � 2 p = g ( p )( v , v ) for v ∈ T p M . Riemannian distance d : M × M → [ 0 , + ∞ ) : � b d ( p , q ) = inf { a � ˙ c ( t ) � c ( t ) dt | c : [ a , b ] → M , c ( a ) = p , c ( b ) = q } Assumption: ( M , d ) is a complete metric space. Covariant derivative: F ′ ( p ) v := ∇ v F ( p ) = ( ∇ Y F )( p ) , v ∈ T p M , where � Y is any vector field on M satisfying v = Y ( p ) . � ∇ is the Riemannian (or Levi-Civita) connection on ( M , g ) . Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 8 / 23

Abstract differential geometry setting for R-Newton Other explicit examples Metric framework M is endowed with a Riemannian metric g : � v � 2 p = g ( p )( v , v ) for v ∈ T p M . Riemannian distance d : M × M → [ 0 , + ∞ ) : � b d ( p , q ) = inf { a � ˙ c ( t ) � c ( t ) dt | c : [ a , b ] → M , c ( a ) = p , c ( b ) = q } Assumption: ( M , d ) is a complete metric space. Covariant derivative: F ′ ( p ) v := ∇ v F ( p ) = ( ∇ Y F )( p ) , v ∈ T p M , where � Y is any vector field on M satisfying v = Y ( p ) . � ∇ is the Riemannian (or Levi-Civita) connection on ( M , g ) . Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 8 / 23

Abstract differential geometry setting for R-Newton Other explicit examples Metric framework M is endowed with a Riemannian metric g : � v � 2 p = g ( p )( v , v ) for v ∈ T p M . Riemannian distance d : M × M → [ 0 , + ∞ ) : � b d ( p , q ) = inf { a � ˙ c ( t ) � c ( t ) dt | c : [ a , b ] → M , c ( a ) = p , c ( b ) = q } Assumption: ( M , d ) is a complete metric space. Covariant derivative: F ′ ( p ) v := ∇ v F ( p ) = ( ∇ Y F )( p ) , v ∈ T p M , where � Y is any vector field on M satisfying v = Y ( p ) . � ∇ is the Riemannian (or Levi-Civita) connection on ( M , g ) . Felipe Álvarez (DIM) A User’s Guide to Riemannian Newton-Type Methods 8 / 23