Chapter 2 Linear Ill-Posed Problems Observations from previous - PowerPoint PPT Presentation

Linear Ill-Posed Problems Michael Moeller Chapter 2 Linear Ill-Posed Problems Observations from previous chapter Ill-Posed Problems in Image and Signal Processing Finite dimensional WS 2014/2015 linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear operators The singular value decomposition Michael Moeller Optimization and Data Analysis Department of Mathematics TU M¨ unchen updated 30.10.2014

Linear Ill-Posed What we have seen so far... Problems • Differentiation : Finding u ( x ) for given Michael Moeller � x u ( y ) dy 0 is ill-posed . Observations from • Inverse heat equation : Finding u ( x , 0 ) for given previous chapter Finite dimensional � π linear operators u ( x , T ) = k ( x , y , T ) f ( y ) dy , Some functional analysis basics 0 ∞ Linear operators in k ( x , y , T ) = 2 e − n 2 T sin ( nx ) sin ( ny ) . � infinite dimensions π Compact linear operators n = 1 The singular value is ill-posed . decomposition • Deconvolution : Finding u ( x ) for given � k ( x − y ) u ( y ) dy Ω with smoothing kernel k is ill-posed . Question Is the inversion of integral operators ill-posed in general? updated 30.10.2014

Linear Ill-Posed Why are problems ill-posed? Problems Michael Moeller We want to study and understand our introductory examples in Observations from more detail. previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear operators The singular value decomposition updated 30.10.2014

Linear Ill-Posed Why are problems ill-posed? Problems Michael Moeller We want to study and understand our introductory examples in Observations from more detail. previous chapter Finite dimensional linear operators Some functional Observation: Our introductory problems can be written as analysis basics Linear operators in infinite dimensions f = Au Compact linear operators for linear operators A : X → Y between Hilbert spaces X , Y . The singular value decomposition updated 30.10.2014

Linear Ill-Posed Why are problems ill-posed? Problems Michael Moeller We want to study and understand our introductory examples in Observations from more detail. previous chapter Finite dimensional linear operators Some functional Observation: Our introductory problems can be written as analysis basics Linear operators in infinite dimensions f = Au Compact linear operators for linear operators A : X → Y between Hilbert spaces X , Y . The singular value decomposition Strategy: Understand finite dimensional case first! updated 30.10.2014

Linear Ill-Posed Finite dimensional linear operators Problems Michael Moeller Linear operators between two finite dimensional Hilbert spaces: Matrices. Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear operators The singular value decomposition updated 30.10.2014

Linear Ill-Posed Finite dimensional linear operators Problems Michael Moeller Linear operators between two finite dimensional Hilbert spaces: Matrices. Observations from previous chapter Finite dimensional Making our life easier: Consider a linear operator from a finite linear operators dimensional Hilbert space into itself: A ∈ R n × n . Some functional analysis basics Linear operators in infinite dimensions Compact linear operators The singular value decomposition updated 30.10.2014

Linear Ill-Posed Finite dimensional linear operators Problems Michael Moeller Linear operators between two finite dimensional Hilbert spaces: Matrices. Observations from previous chapter Finite dimensional Making our life easier: Consider a linear operator from a finite linear operators dimensional Hilbert space into itself: A ∈ R n × n . Some functional analysis basics Linear operators in infinite dimensions Compact linear Corresponding finite dimensional linear inverse problem: Find operators u from given The singular value decomposition f = Au updated 30.10.2014

Linear Ill-Posed Finite dimensional linear operators Problems Michael Moeller Linear operators between two finite dimensional Hilbert spaces: Matrices. Observations from previous chapter Finite dimensional Making our life easier: Consider a linear operator from a finite linear operators dimensional Hilbert space into itself: A ∈ R n × n . Some functional analysis basics Linear operators in infinite dimensions Compact linear Corresponding finite dimensional linear inverse problem: Find operators u from given The singular value decomposition f = Au Making our life even easier: A is symmetric and positive definite. updated 30.10.2014

Linear Ill-Posed Finite dimensional linear operators Problems Michael Moeller Symmetric positive definite A ∈ R n × n : Observations from previous chapter A = VSV T , Finite dimensional linear operators Some functional with analysis basics Linear operators in • diagonal matrix S , S i , i = λ i eigenvalues, infinite dimensions • λ 1 ≥ ... ≥ λ n > 0, Compact linear operators • V orthonormal matrix of eigenvectors. The singular value decomposition 1 Assume scaling: λ 1 = 1. Condition number κ = λ n . updated 30.10.2014

Linear Ill-Posed Finite dimensional linear operators Problems Michael Moeller Assume f = Au , f δ = Au δ , with � f δ − f � ≤ δ : Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear operators The singular value decomposition updated 30.10.2014

Linear Ill-Posed Finite dimensional linear operators Problems Michael Moeller Assume f = Au , f δ = Au δ , with � f δ − f � ≤ δ : u δ − u = VS − 1 V T ( f δ − f ) ⇒ � u δ − u � = � VS − 1 V T ( f δ − f ) � Observations from previous chapter = � S − 1 V T ( f δ − f ) � Finite dimensional linear operators ≤ 1 Some functional � V T ( f δ − f ) � analysis basics λ n Linear operators in infinite dimensions = δ = κδ Compact linear λ n operators The singular value decomposition updated 30.10.2014

Linear Ill-Posed Finite dimensional linear operators Problems Michael Moeller Assume f = Au , f δ = Au δ , with � f δ − f � ≤ δ : u δ − u = VS − 1 V T ( f δ − f ) ⇒ � u δ − u � = � VS − 1 V T ( f δ − f ) � Observations from previous chapter = � S − 1 V T ( f δ − f ) � Finite dimensional linear operators ≤ 1 Some functional � V T ( f δ − f ) � analysis basics λ n Linear operators in infinite dimensions = δ = κδ Compact linear λ n operators The singular value decomposition → Noise amplification: reciprocal of smallest eigenvalue! → Continuous dependence on the data! → Well-posed, but for small λ n ill-conditioned! → In infinite dimensions: infinitely many λ n → 0! updated 30.10.2014

Linear Ill-Posed Finite dimensional linear operators Problems Michael Moeller Question What can we do against the instability? Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear operators The singular value decomposition updated 30.10.2014

Linear Ill-Posed Finite dimensional linear operators Problems Michael Moeller Question What can we do against the instability? Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Idea: Linear operators in infinite dimensions • Approximate A by A α = A + α I with α > 0. Compact linear • The smallest eigenvalue is λ n + α > α . operators The singular value • Approximate the solution to Au = f for given noisy data f δ decomposition by u α = A − 1 α f δ . Computation on the board. updated 30.10.2014

Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Linear inverse problems in Some functional analysis basics infinite dimensions. Linear operators in infinite dimensions Compact linear operators The singular value decomposition updated 30.10.2014

Linear Ill-Posed Some basics... Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear operators The singular value decomposition updated 30.10.2014

Linear Ill-Posed Some basics... Problems Michael Moeller Definition: Banach space A normed vector space X which is complete is called a Banach Observations from previous chapter space . Being complete means that every Cauchy sequence Finite dimensional converges in X . linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear operators The singular value decomposition updated 30.10.2014