Heuristic parameter choice in Tikhonov method from minimizers of the - PowerPoint PPT Presentation

Heuristic parameter choice in Tikhonov method from minimizers of the quasi-optimality function Toomas Raus and Uno H¨ amarik University of Tartu, Estonia 31.10.2017, Rio de Janeiro U. H¨ amarik (Tartu) Heuristic parameter choice 31.10.2017, Rio de Janeiro 1 / 21

Contents 1 Problem 2 Rules for the choice of the regularization parameter 3 Local minimum points of the function ψ Q ( α ) 4 Restricted set L ∗ min of the local minimizers of ψ Q ( α ) 5 Choice of α from the set L ∗ min U. H¨ amarik (Tartu) Heuristic parameter choice 31.10.2017, Rio de Janeiro 2 / 21

Problem A ∈ L ( H , F ) , f ∗ ∈ R ( A ) , Au = f ∗ , (1) f ∗ , f - exact and noisy data. Range R ( A ) non-closed, kernel N ( A ) non-trivial. Tikhonov method, using f ∗ , f : α = ( α I + A ∗ A ) − 1 A ∗ f ∗ , u α = ( α I + A ∗ A ) − 1 A ∗ f u + Problem: how to choose the regularization parameter α > 0? Denote � + � . � u + � u α − u + � � � � e 1 ( α ) := α − u ∗ (2) α The aim: find rule R, choosing the parameter α R with the property (’pseudooptimality’ property) � u α R − u ∗ � ≤ const min α> 0 e 1 ( α ) U. H¨ amarik (Tartu) Heuristic parameter choice 31.10.2017, Rio de Janeiro 3 / 21

Quasioptimality criterion Quasioptimality criterion: α Q := argmin α> 0 ψ Q ( α ), where � = α − 1 � A ∗ ( Au 2 ,α − f ) � , � du α � � ψ Q ( α ) := α d α u 2 ,α = ( α I + A ∗ A ) − 1 ( α u α + A ∗ f ). We search the regularization parameter from the set Ω = { α j : α j = q α j − 1 , j = 1 , 2 , ..., M , 0 < q < 1 } , where α 0 , M , q are given. We show that at least one of local minimizers from the set Ω is psudooptimal and we show how to find it. U. H¨ amarik (Tartu) Heuristic parameter choice 31.10.2017, Rio de Janeiro 4 / 21

Rules for the choice of α , if δ ≥ � f − f ∗ � is known 1) Discrepancy principle : b 1 δ ≤ � Au α − f � ≤ b 2 δ, b 1 ≥ 1 . 2) Modified discrepancy principle : ( Au α − f , Au 2 ,α − f ) 1 / 2 = δ, u 2 ,α = ( α I + A ∗ A ) − 1 ( α u α + A ∗ f ) . 3) Monotone error rule (ME-rule) ( Au α − f , Au 2 ,α − f ) = δ � Au 2 ,α − f � gives α ME ≥ α opt := argmin � u α − u ∗ � . We recommend α MEe := 0 . 4 α ME ; typically � u α MEe − u ∗ � / � u α ME − u ∗ � ∈ (0 . 7 , 0 . 9) . U. H¨ amarik (Tartu) Heuristic parameter choice 31.10.2017, Rio de Janeiro 5 / 21

Rules for the choice of α , if δ ≥ � f − f ∗ � is unknown 1) The quasi-optimality criterion: α = α Q is the global minimizer of the function � � du α � = α − 1 � A ∗ ( Au 2 ,α − f ) � . � � ψ Q ( α ) = α (3) � � d α � 2) The Hanke-Raus rule: α = α HR is the global minimizer of the function ψ HR ( α ) = α − 1 / 2 ( A α − f , Au 2 ,α − f ) 1 / 2 . 3) L-curve rule: on graph with log-log scale, on x -axis � Au α − f � and on y -axis � u α � the corner point is used. 4) Reginska’s rule: global minimum point of the function ψ RE ( α ) = � Au α − f � � u α � τ , τ ≥ 1 . 5) HME-rule: α = α HME is chosen as the global minimizer of the function ψ HME ( α ) = α − 1 / 2 ( Au α − f , Au 2 ,α − f ) . � Au 2 ,α − f � U. H¨ amarik (Tartu) Heuristic parameter choice 31.10.2017, Rio de Janeiro 6 / 21

Parameters set, regularization need We use the set of parameters Ω = { α j : α j = q α j − 1 , j = 1 , 2 , ..., M , 0 < q < 1 } , (4) where α 0 , q , α M are given. We search local minimizer of ψ Q ( α ) in the interval [max ( α M , λ min ) , α 0 ], where λ min is the minimal eigenvalue of the matrix A T A of the discretized problem. We say that the discretized problem Au = f do not need regularization if e 1 ( λ min ) = α ∈ Ω ,α ≥ λ min e 1 ( α ) . min If λ min > α M and the discretized problem do not need regularization then α M is the proper parameter while then it is easy to show the error estimate � u α M − u ∗ � ≤ e 1 ( α M ) ≤ 2 min α ∈ Ω e 1 ( α ) . Searching the parameter from the interval [max ( α M , λ min ) , α 0 ] means the a priori assumption that the discretized problem needs regularization. U. H¨ amarik (Tartu) Heuristic parameter choice 31.10.2017, Rio de Janeiro 7 / 21

Test problems Hansen’s 10 test problems: Baart, Deriv2,Foxgood,Gravity,Heat,Ilaplace, Phillips,Shaw,Spikes,Wing. The problems are scaled in such a way that the 2-norms of A and f are 1. The dicretization number n = 100. Relative noise levels: δ rel := � f − f ∗ � / � f ∗ � = 10 − j , j = 1 , 2 , ..., 6 . Noisy right-hand side: f = f ∗ + δ rel � f ∗ � � ξ � − 1 ξ , where components ξ i have standard normal distribution. For each noise level we considered 20 runs. Sequence of the parameters: α 0 = 1 , α M = 10 − 18 , q = 0 . 95. case p = 0. Original problems. case p = 2. Problems, where the exact solution u ∗ is replaced by A T Au ∗ . Let λ min be the minimal eigenvalue of the matrix A T A . Tables below show for different rules R the error ratios E = � u α R − u ∗ � � u α R − u ∗ � � u α ∗ − u ∗ � = min α ∈ Ω � u α − u ∗ � . U. H¨ amarik (Tartu) Heuristic parameter choice 31.10.2017, Rio de Janeiro 8 / 21

Comparison of heuristic rules Table: Averages of error ratios E and failure % (in parenthesis) for heuristic rules, p = 0 Problem Λ Quasiopt. HR HME Reginska Baart 1666 1.54 2.58 2.52 1.32 Deriv2 16 1.08 2.07 1.72 35.19 (3.3) Foxgood 210 1.57 8.36 7.71 36.94 (10.8) Gravity 4 1.13 2.66 2.32 20.49 (0.8) 4 ∗ 10 29 Heat > 100 (66.7) 1.64 1.48 23.40 (4.2) Ilaplace 16 1.24 1.94 1.81 1.66 Phillips 9 1.09 2.27 1.91 > 100 (44.2) Shaw 290 1.43 2.34 2.23 1.80 Spikes 1529 1.01 1.03 1.03 1.01 Wing 9219 1.40 1.51 1.51 1.18 Λ = max λ k > max ( α M ,λ n ) λ k /λ k +1 of consecutive eigenvalues λ 1 ≥ λ 2 ≥ ... ≥ λ n of the matrix A T A in the interval [max ( α M , λ n ) , 1]. U. H¨ amarik (Tartu) Heuristic parameter choice 31.10.2017, Rio de Janeiro 9 / 21

Local minimum points of the function ψ Q ( α ) Lemma 1 The function ψ Q ( α ) has the estimate (see (2) for notation e 1 ( α ) ) � u + � + � u α − u + � . � � � � ψ Q ( α ) ≤ e 1 ( α ) = α − u ∗ (5) α Remark 1 Note that lim α →∞ ψ Q ( α ) = 0 , but lim α →∞ e 1 ( α ) = � u ∗ � . Therefore in the case of too large α 0 this α 0 may be global (or local) minimizer of the function ψ Q ( α ) . We recommend to take α 0 = c � A ∗ A � , c ≤ 1 or to minimize the function ˜ ψ Q ( α ) := (1 + α/ � A ∗ A � ) ψ Q ( α ) instead of ψ Q ( α ) . Lemma 2 Denote ψ QD ( α ) = (1 − q ) − 1 � u α − u q α � . Then it holds ψ Q ( α ) ≤ ψ QD ( α ) ≤ q − 1 ψ Q ( q α ) . U. H¨ amarik (Tartu) Heuristic parameter choice 31.10.2017, Rio de Janeiro 10 / 21

Parameter α k , 0 ≤ k ≤ M − 1 is the local minimum point of the sequence ψ Q ( α k ), if ψ Q ( α k ) < ψ Q ( α k +1 ) and in case k > 0 there exists index j ≥ 1 such, that ψ Q ( α k ) = ψ Q ( α k − 1 ) = ... = ψ Q ( α k − j +1 ) < ψ Q ( α k − j ) . Parameter α M is the local minimum point if there ∃ j ≥ 1 so, that ψ Q ( α M ) = ψ Q ( α M − 1 ) = ... = ψ Q ( α M − j +1 ) < ψ Q ( α M − j ) . Denote the set of local minimum points: � � α ( k ) min : α (1) min > α (2) min > ... > α ( K ) L min = . min Parameter α k is the local maximum point of the sequence ψ Q ( α k ) if ψ Q ( α k ) > ψ Q ( α k +1 ) and there exists index j ≥ 1 so, that ψ Q ( α k ) = ψ Q ( α k − 1 ) = ... = ψ Q ( α k − j +1 ) > ψ Q ( α k − j ) . We denote by α ( k ) max the local maximum point between the local minimum points α ( k +1) and α ( k ) min , 1 ≤ k ≤ K − 1. Denote α (0) max = α 0 , α ( K ) max = α M . min Then by the construction α (0) max ≥ α (1) min > α (1) max > ... > α ( K − 1) > α ( K ) min ≥ α ( K ) max . max U. H¨ amarik (Tartu) Heuristic parameter choice 31.10.2017, Rio de Janeiro 11 / 21

Theorem 3 Local minimum points of the function ψ Q ( α ) have estimates: 1 α ∈ L min � u α − u ∗ � ≤ q − 1 C min α M ≤ α ≤ α 0 e 1 ( α ) , min where � α 0 � � � α ( k ) C := 1 + max max T min , α j ≤ 1 + c q ln , α M 1 ≤ k ≤ K α j ∈ Ω ,α ( k ) max ≤ α j ≤ α ( k − 1) max T ( α, β ) := � u α − u β � q − 1 − 1 / ln q − 1 → 1 if q → 1 . � � , c q := ψ Q ( β ) 2 Let u ∗ = ( A ∗ A ) p / 2 v, � v � ≤ ρ , 0 < p ≤ 2 and α 0 = 1 . If δ 0 := √ α M ≤ � f − f ∗ � , then α ∈ L min � u α − u ∗ � ≤ c p ln � f − f ∗ � 1 p p +1 | ln � f − f ∗ �| � f − f ∗ � p +1 . min ρ δ 0 U. H¨ amarik (Tartu) Heuristic parameter choice 31.10.2017, Rio de Janeiro 12 / 21