Minimization strategy for choice of the regularization parameter in - PowerPoint PPT Presentation

Introduction Proposals for rules Numerical examples Minimization strategy for choice of the regularization parameter in case of roughly given or unknown noise level Uno Hämarik, Reimo Palm, Toomas Raus Vienna, July 23, 2009 Uno Hämarik, Reimo Palm, Toomas Raus Minimization strategy for choice of the regularization parameter in case of roughly given or unknown noise level

Introduction Proposals for rules Numerical examples The problem ◮ We consider an operator equation Au = f 0 , where A is a linear bounded operator between real Hilbert spaces, f 0 ∈ R ( A ) ⇒ ∃ solution u ∗ ∈ H . ◮ Instead of exact data f 0 , noisy data f are available. ◮ Knowledge of � f 0 − f � : ◮ Case 1: exact noise level δ : � f 0 − f � ≤ δ ◮ Case 2: approximate noise level δ : lim � f 0 − f � /δ ≤ C as δ → 0 ◮ Case 3: no information about � f 0 − f � Uno Hämarik, Reimo Palm, Toomas Raus Minimization strategy for choice of the regularization parameter in case of roughly given or unknown noise level

Introduction Proposals for rules Numerical examples Methods Let D = A ∗ in case A � = A ∗ and D = I in case A = A ∗ ≥ 0. ◮ (Iterated) Tikhonov method ( D = A ∗ ) and (iterated) Lavrentiev method ( D = I ): u α = u m ,α , where u k ,α = ( α I + DA ) − 1 ( α u k − 1 ,α + Df ) , k = 1, . . . , m . ◮ Landweber method: u n = u n − 1 − µ D ( Au n − 1 − f ) , µ ∈ ( 0 , 1 / � DA � ) , n = 1, 2, . . . We realized Landweber method by operator iterations n = m k , k = 1 , 2 , . . . , u n = ( I − DAC k ) u 0 + C k Df , m − 1 � ( I − DAC k − 1 ) j , C k = C k − 1 C 0 = µ I , k = 1 , 2 , . . . j = 0 Uno Hämarik, Reimo Palm, Toomas Raus Minimization strategy for choice of the regularization parameter in case of roughly given or unknown noise level

Introduction Proposals for rules Numerical examples Rules for choice of α in (iterated) Tikhonov method ◮ Case 1: δ with � f − f 0 � ≤ δ . Rule D : � r m ,α D � = δ , where r m ,α = Au m ,α − f . Rule MEe : 1) find by the monotone error rule α ME as solution of equation ( r m ,α , r m + 1 ,α ) / � r m + 1 ,α � = δ ; 2) take α MEe = min ( 0 . 53 α ME , 0 . 6 α 1 . 06 ME ) . ◮ Case 2: approximate δ is known with � f − f 0 � /δ ≤ const ( δ → 0). Rule DM : 1) find α as maximal solution of equation √ α � u m ,α − u m + 1 ,α � = b ( m + 1 ) m + 1 / m m · δ , b = const; 2) fix s ∈ ( 0 , 1 ) , q ∈ ( 0 , 1 ) and find α ( δ ) = argmin { Φ( α ) /α s / 2 , α ∈ [ α, 0 . 4 m + 0 . 6 ] } , where α � A � 2 ) 1 / ( 2 m ) . In computations we used Φ( α ) = � u α − u q α � ( 1 + b = s = 0 . 01, q = 0 . 9 and if the first equation had no solution, then the largest local minimum of √ α � u m ,α − u m + 1 ,α � was taken as α . Uno Hämarik, Reimo Palm, Toomas Raus Minimization strategy for choice of the regularization parameter in case of roughly given or unknown noise level

Introduction Proposals for rules Numerical examples Rules for Case 3 in (iterated) Tikhonov method ◮ Hanke-Raus rule : α HR is the global minimizer of the function

Introduction Proposals for rules Numerical examples ◮ Rules QO1 , D1 , R21 and DR21 choose the parameter by the largest local minimum in functions ϕ QN ( α ) α 1 / 3 , � r m ,α � α − 1 / 6 , d R2 ( α ) α − 1 / 6 and � � r m ,α � d R2 ( α ) α − 1 / 6 . ◮ Rule HR2 chooses the parameter as the global minimizer of “ d R2 ( α ) “ d R2 ( α ) ” τ ” τ / √ α. with 1 − ϕ HR2 ,τ ( α ) = d HR ( α ) d R2 ( α ) d MD ( α ) d MD ( α ) τ = 0 . 15. ◮ Rule QHR2 chooses local minimizer of the function ϕ QN ( α ) κ ( α ) for which the function ϕ HR2 ,τ ( α ) with τ = 0 . 04 is minimal. ◮ Rule QOHR chooses local minimizer of the function ϕ QN ( α ) κ ( α ) for which the function d MD ( α ) / √ α is minimal. Uno Hämarik, Reimo Palm, Toomas Raus Minimization strategy for choice of the regularization parameter in case of roughly given or unknown noise level

Introduction Proposals for rules Numerical examples Rules for choice of α in (iterated) Lavrentiev method ◮ Case 1: δ with � f − f 0 � ≤ δ . Rules MD , MEa choose α ’s from equations � r m + 1 ,α � = 1 . 143 δ and � r m + 1 ,α � 2 � r m + 2 ,α � = 1 . 364 δ , respectively. Rules MEn , MEl choose α and α i = 1 . 2 − i from ( r 2 m + 1 ,α , r 1 , 0 . 2 α ) equations ( r 2 m + 2 ,α , r 2 , 0 . 2 α ) 1 / 2 = 1 . 096 δ , and ( u m ,α i − u m ,α i − 1 , u m ,α 5 + i − u m ,α 5 + i − 1 ) q − 1 · m + 1 α i · m 2 ≈ 1 . 004 δ, , � u m + 1 ,α 5 + i , u m + 1 ,α 4 + i � respectively. ◮ Case 2: approximate δ is known with � f − f 0 � /δ ≤ const ( δ → 0). Rule DM : 1) find α as minimal solution of equation ( r α, 1 , Ar α, 2 ) / √ α = 1 √ 2 m + 3 · δ ; 2) fix s ∈ ( 0 , 1 ) , q ∈ ( 0 , 1 ) and find α ( δ ) = argmin { Φ( α ) /α s , α ∈ [ α, m ] } , where Φ( α ) = � r α, 1 � · ( r α, 1 , r α, 2 ) 1 / 2 / ( r α, 2 , r α, 3 ) 1 / 2 /α in case m = 1, � A � ) 0 . 005 in case m ≥ 2. α and Φ( α ) = � u α − u q α � ( 1 + Uno Hämarik, Reimo Palm, Toomas Raus Minimization strategy for choice of the regularization parameter in case of roughly given or unknown noise level

Introduction Proposals for rules Numerical examples Case 3 for (iterated) Lavrentiev method ◮ Rules QOC , QOmC minimize the functions � A � ) 0 . 005 and α � u α − u q α � ( 1 + � r m ,α � · ( r m ,α , r m + 1 ,α ) 1 / 2 / ( r m + 1 ,α , r m + 2 ,α ) 1 / 2 on the interval [ α, m ] , where α is the largest α , for which the values of these functions are 1 . 4 times larger than their values at the minimum point. Uno Hämarik, Reimo Palm, Toomas Raus Minimization strategy for choice of the regularization parameter in case of roughly given or unknown noise level

Introduction

Introduction Proposals for rules Numerical examples Convergence and error estimate for Rule DM (Case 2) Let the parameter r ( δ ) = α − 1 in γ = m times iterated Tikhonov or Lavrentiev method or r ( δ ) = n in Landweber method with γ = µ be chosen according to Rule DM. If lim � f 0 − f � /δ ≤ C as δ → 0, then � u r − u ∗ � → 0 and in case b ≥ 1 the following error estimates hold: 1. If � f 0 − f � ≤ max { δ, ˜ δ ( r ( δ )) } , where ˜ δ ( r ( δ )) := 1 2 � r m + 1 ,α ( δ ) � in (iterated) Tikhonov or Lavrentiev method, ˜ δ ( r ( δ )) = 1 2 � Au n ( δ ) − f � in Landweber method, then 1 � u r ( δ ) − u ∗ � ≤ C ′ r ≥ 0 Ψ( r ) , Ψ( r ) = e 0 1 − s /σ inf r + γ r max { δ, � f 0 − f �} , e 0 r = � u 0 r − u ∗ � , u 0 r is the approximation at the exact data. 2. If max { δ, ˜ 2 ˜ δ ( r ( δ )) } < � f 0 − f � ≤ 1 δ ( 1 ) , then δ ( r ( δ ))) σ/ s inf � u r ( δ ) − u ∗ � ≤ C ′′ ( � f 0 − f � / ˜ r ≥ 0 Ψ( r ) . Uno Hämarik, Reimo Palm, Toomas Raus Minimization strategy for choice of the regularization parameter in case of roughly given or unknown noise level

Introduction Proposals for rules Numerical examples Test problems ◮ 10 test problems of P. C. Hansen. Typically integral equations of the first kind, arising from various applications: inverse heat equation, gravity surveying, inverse Laplace transform etc. ◮ Supposable noise levels: δ = 0 . 5, 10 − 1 , 10 − 2 , . . . , 10 − 6 . Actual noise level is d δ where d = 1, 10. Each problem was solved 10 times. ◮ The discretized problems (discretization parameter = 100) were solved using different parameter choice rules. ◮ In the following tables we present averages of error ratios � u r − u ∗ � / e opt (over δ ’s and 10 runs) in case δ = d � f δ − f � , where e opt = min {� u r − u ∗ � : r ≥ 0 } and d = 1, 10 ( d > 1 corresponds to overestimation of noise level); e D and e D,10 correspond to α D in cases d = 1 and d = 10 etc. Uno Hämarik, Reimo Palm, Toomas Raus Minimization strategy for choice of the regularization parameter in case of roughly given or unknown noise level

Introduction Proposals for rules Numerical examples Results for Tikhonov method Problem e D e D,10 e MEe e MEe,10 e DM e DM,10 e HR e QN e QOC baart 1.41 3.21 1.38 2.46 1.72 2.60 2.76 1.56 1.89 deriv2 1.20 3.44 1.05 1.97 1.14 1.50 1e3 1.87 1.14 foxgood 1.38 11.2 1.37 6.98 2.09 5.12 8.16 2.18 2.18 gravity 1.15 4.42 1.09 2.71 1.11 2.08 2.81 1.13 1.13 heat 1.06 2.76 1.03 1.82 1.17 1.36 1.70 1e5 1.25 ilaplace 1.26 2.58 1.17 2.00 1.21 1.67 2.05 1.19 1.19 phillips 1.03 4.20 1.04 2.43 1.09 1.78 1e5 1.08 1.08 shaw 1.31 3.20 1.26 2.43 1.48 2.25 2.57 1.44 1.47 spikes 1.02 1.08 1.02 1.06 1.04 1.07 1.07 1.04 1.05 wing 1.19 1.50 1.18 1.47 1.48 1.55 1.56 1.43 1.79 Mean 1.20 3.75 1.16 2.53 1.35 2.10 1e4 1e4 1.42 Uno Hämarik, Reimo Palm, Toomas Raus Minimization strategy for choice of the regularization parameter in case of roughly given or unknown noise level