Inverse Problem: A . . . Enter Soft Constraints Regularization How to Determine the . . . Why Curvature in L-Curve: Analysis of the . . . Additional Invariance . . . Combining Soft Constraints Main Result Acknowledgments Proof Anibal Sosa, Martine Ceberio, and Vladik Kreinovich Home Page Title Page Cyber-ShARE Center University of Texas at El Paso ◭◭ ◮◮ 500 W. University El Paso, TX 79968, USA ◭ ◮ usosaaguirre@miners.utep.edu Page 1 of 14 mceberio@utep.edu, vladik@utep.edu Go Back Full Screen Close Quit
Inverse Problem: A . . . Enter Soft Constraints 1. Inverse Problem: A Brief Reminder Regularization • In science & engineering, we are interested in the state How to Determine the . . . of the world, i.e., in the values of different quantities. Analysis of the . . . Additional Invariance . . . • Some of these quantities we can directly measure, but Main Result many quantities are difficult to measure directly. Acknowledgments • For example, in geophysics, we are interested in the Proof density at different depths and different locations. Home Page • In principle, we can drill a borehole and directly mea- Title Page sure these properties, but this is very expensive. ◭◭ ◮◮ • To find the values of such difficult-to-measure quanti- ◭ ◮ ties q = ( q 1 , . . . , q n ), we: Page 2 of 14 – measure auxiliary quantities a = ( a 1 , . . . , a m ) re- Go Back lated to q i by a known dependence a i = f i ( q 1 , . . . , q n ), Full Screen – and then reconstruct the values q j from these mea- surement results. Close Quit
Inverse Problem: A . . . Enter Soft Constraints 2. Enter Soft Constraints Regularization • Objective: describe the constraint that the values q j How to Determine the . . . are consistent with the observations a i . Analysis of the . . . Additional Invariance . . . • Assumption: measurement errors a i − f i ( q 1 , . . . , q n ) are Main Result indep. normal variables with 0 mean and same σ 2 . Acknowledgments • Resulting constraint: s ≤ s 0 , where Proof m � Home Page def ( a i − f i ( q 1 , . . . , q n )) 2 . = s Title Page i =1 ◭◭ ◮◮ • Fact: for each s 0 , there is a certain probability that ◭ ◮ this constraint will be violated. Page 3 of 14 • Soft constraints: such constraints are called soft . Go Back • For convenience: this constraint is sometimes described def in a log scale, as x ≤ x 0 , where x = ln( s ). Full Screen Close Quit
Inverse Problem: A . . . Enter Soft Constraints 3. Regularization Regularization • Methods for taking additional regularity constraints How to Determine the . . . into account are known as regularization methods . Analysis of the . . . Additional Invariance . . . • Example: In geophysics, the density values at nearby Main Result locations are usually close to each other: q j − q j ′ ≈ 0. Acknowledgments • Assumption: differences q j − q j ′ are indep. and normally Proof distributed with 0 mean and the same σ 2 d . Home Page = � def ( q j − q j ′ ) 2 . • Resulting constraint: t ≤ t 0 , where t Title Page ( j,j ′ ) ◭◭ ◮◮ def • In log scale: y ≤ y 0 , where y = ln( t ). ◭ ◮ • How to combine constraints: e.g., we can use the Max- Page 4 of 14 imum Likelihood method. Go Back • Result: we find the values q j that minimize the sum Full Screen s + λ · t , where λ depends on the variances σ 2 and σ 2 d . Close Quit
Inverse Problem: A . . . Enter Soft Constraints 4. How to Determine the Parameter λ Regularization • Fact: for each λ , we can find q j ( λ ), and, based on this How to Determine the . . . solution, compute x ( λ ) and y ( λ ). Analysis of the . . . Additional Invariance . . . • Question: what value λ shall we choose? Main Result • Often: the curve ( x ( λ ) , y ( λ )) has a turning point (is Acknowledgments L-shaped ). Proof • In this case: it is reasonable to select this turning point. Home Page • How to describe it: it is a point where the absolute Title Page value | C | of the curvature C is the largest: ◭◭ ◮◮ x ′′ · y ′ − y ′′ · x ′ def C = (( x ′ ) 2 + ( y ′ ) 2 ) 3 / 2 . ◭ ◮ Page 5 of 14 • Fact: this approach often works well. Go Back • Natural question: explain why curvature works well. Full Screen • What we do: we show that reasonable properties select a class of functions that include curvature. Close Quit
Inverse Problem: A . . . Enter Soft Constraints 5. Analysis of the Problem: Scale-Invariance Regularization • The numerical values of each quantity depend on the How to Determine the . . . selection of a measuring unit a . Analysis of the . . . Additional Invariance . . . • If we change a to a new measuring unit c a times smaller, Main Result then a i and a i − f i ( q 1 , . . . , q n ) get multiplied by c a . Acknowledgments � n ( a i − f i ( q 1 , . . . , q n )) 2 get multiplied by c 2 • So, s = a , Proof i =1 Home Page def = ln( c 2 and x = ln( s ) changes to x + ∆ x , where ∆ x a ). Title Page • If we change a measuring unit q by a new one c q times ◭◭ ◮◮ smaller, then q j and q j − q j ′ get multiplied by c 2 q . • Also t = � ( q j − q j ′ ) 2 , and y = ln( t ) get multiplied by ◭ ◮ Page 6 of 14 c 2 q , and y = ln( t ) changes to y + ∆ y Go Back • Under these changes x ( λ ) → x ( λ ) + ∆ x and y ( λ ) → Full Screen y ( λ ) + ∆ y , and the curvature does not change. Close Quit
Inverse Problem: A . . . Enter Soft Constraints 6. Additional Invariance and Our Main Idea Regularization • Instead of the original parameter λ , we can use a new How to Determine the . . . parameter µ for which λ = g ( µ ). Analysis of the . . . Additional Invariance . . . • This re-scaling of a parameter does not change the Main Result curve itself and thus, does not change its curvature. Acknowledgments • Our idea: to describe all the functions which are in- Proof variant with respect to both types of re-scalings. Home Page • By a parameter selection criterion we mean a function Title Page F ( x, y, x ′ , y ′ , x ′′ , y ′′ ) of six variables. ◭◭ ◮◮ • F is scale-invariant if for all values ∆ x and ∆ y , ◭ ◮ F ( x + ∆ x , y + ∆ y , x ′ , y ′ , x ′′ , y ′′ ) = F ( x, y, x ′ , y ′ , x ′′ , y ′′ ); Page 7 of 14 • F is invariant w.r.t. parameter re-scaling if for every Go Back function g ( z ) , for � x ( µ ) = x ( g ( µ )) , � y ( µ ) = y ( g ( µ )) , Full Screen x ′ , � y ′ , � x ′′ , � y ′′ ) = F ( x, y, x ′ , y ′ , x ′′ , y ′′ ) . F ( � x, � y, � Close Quit
Inverse Problem: A . . . Enter Soft Constraints 7. Main Result Regularization Main Result. A parameter selection criterion is scale- How to Determine the . . . invariant and invariant w.r.t. parameter re-scaling if and Analysis of the . . . only if it has the form Additional Invariance . . . � � C ( x, y, x ′ , y ′ , x ′′ , y ′′ ) , y ′ Main Result F ( x, y, x ′ , y ′ , x ′′ , y ′′ ) = f Acknowledgments x ′ Proof for some function f ( C, z ) , where Home Page x ′′ · y ′ − y ′′ · x ′ def Title Page C = (( x ′ ) 2 + ( y ′ ) 2 ) 3 / 2 . ◭◭ ◮◮ ◭ ◮ Comment. Once a criterion is selected, for each problem, Page 8 of 14 we use the value λ for which the value Go Back F ( x ( λ ) , y ( λ ) , x ′ ( λ ) , y ′ ( λ ) , x ′′ ( λ ) , y ′′ ( λ )) Full Screen is the largest. Close Quit
Inverse Problem: A . . . Enter Soft Constraints 8. Acknowledgments Regularization This work was supported in part: How to Determine the . . . Analysis of the . . . • by the National Science Foundation grants HRD-0734825 Additional Invariance . . . and DUE-0926721, Main Result • by Grant 1 T36 GM078000-01 from the National Insti- Acknowledgments tutes of Health, and Proof Home Page • by UTEP’s Computational Science program. Title Page ◭◭ ◮◮ ◭ ◮ Page 9 of 14 Go Back Full Screen Close Quit
Inverse Problem: A . . . Enter Soft Constraints 9. Proof Regularization • For each tuple ( x, y, x ′ , y ′ , x ′′ , y ′′ ), by taking ∆ x = − x How to Determine the . . . and ∆ y = − y , we conclude that Analysis of the . . . Additional Invariance . . . F ( x, y, x ′ , y ′ , x ′′ , y ′′ ) = F (0 , 0 , x ′ , y ′ , x ′′ , y ′′ ) . Main Result • Thus, we conclude that Acknowledgments Proof F ( x, y, x ′ , y ′ , x ′′ , y ′′ ) = F 0 ( x ′ , y ′ , x ′′ , y ′′ ) , Home Page def where we denoted F 0 ( x ′ , y ′ , x ′′ , y ′′ ) = F (0 , 0 , x ′ , y ′ , x ′′ , y ′′ ). Title Page ◭◭ ◮◮ • So, we conclude that the value of the parameter selec- tion criterion does not depend on x and y at all. ◭ ◮ • In terms of the function F 0 , invariance w.r.t. parameter Page 10 of 14 x ′ , � y ′ , � x ′′ , � y ′′ ) = F 0 ( x ′ , y ′ , x ′′ , y ′′ ) . re-scaling means that F 0 ( � Go Back • Re-scaling means that we go from the original function Full Screen x ( λ ) to the new function � x ( µ ) = x ( g ( µ )). Close Quit
Recommend
More recommend
