Statistical Data Processing under Interval Uncertainty: Algorithms - PowerPoint PPT Presentation

Statistical Data Processing under Interval Uncertainty: Algorithms and Title Page ◭◭ ◮◮ Computational Complexity ◭ ◮ Vladik Kreinovich Page 1 of 48 Go Back Department of Computer Science University of Texas at El Paso Full Screen El Paso, Texas 79968, USA email vladik@utep.edu Close http://www.cs.utep.edu/vladik Quit http://www.cs.utep.edu/interval-comp

1. General Problem of Data Processing under Uncer- tainty • Indirect measurements: way to measure y that are are difficult (or even impossible) to measure directly. • Idea: y = f ( x 1 , . . . , x n ) Title Page x 1 � ◭◭ ◮◮ ✲ x 2 � y = f ( � x 1 , . . . , � x n ) � f ◭ ◮ ✲ ✲ · · · Page 2 of 48 x n � ✲ Go Back Full Screen • Problem: measurements are never 100% accurate: � x i � = x i (∆ x i � = 0) hence Close Quit � y = f ( � x 1 , . . . , � x n ) � = y = f ( x 1 , . . . , y n ) . def • Question: what are bounds on ∆ y = � y − y ?

2. Probabilistic and Interval Uncertainty ∆ x 1 ✲ ∆ x 2 ∆ y f ✲ ✲ . . . ∆ x n ✲ Title Page ◭◭ ◮◮ • Traditional approach: we know probability distribution ◭ ◮ for ∆ x i (usually Gaussian). Page 3 of 48 • Where it comes from: calibration using standard MI. Go Back • Problem: calibration is not possible in: Full Screen – fundamental science Close – manufacturing Quit • Solution: we know upper bounds ∆ i on | ∆ x i | hence x i ∈ [ � x i − ∆ i , � x i + ∆ i ] .

3. Interval Computations: A Problem x 1 ✲ x 2 y = f ( x 1 , . . . , x n ) f ✲ ✲ · · · x n ✲ Title Page ◭◭ ◮◮ • Given: an algorithm y = f ( x 1 , . . . , x n ) and n intervals ◭ ◮ x i = [ x i , x i ]. Page 4 of 48 • Compute: the corresponding range of y : Go Back [ y, y ] = { f ( x 1 , . . . , x n ) | x 1 ∈ [ x 1 , x 1 ] , . . . , x n ∈ [ x n , x n ] } . Full Screen • Fact: NP-hard even for quadratic f . Close • Challenge: when are feasible algorithm possible? Quit • Challenge: when computing y = [ y, y ] is not feasible, find a good approximation Y ⊇ y .

4. Alternative Approach: Maximum Entropy • Situation: in many practical applications, it is very difficult to come up with the probabilities. • Traditional engineering approach: use probabilistic tech- niques. Title Page • Problem: many different probability distributions are ◭◭ ◮◮ consistent with the same observations. ◭ ◮ • Solution: select one of these distributions – e.g., the Page 5 of 48 one with the largest entropy. Go Back • Example – single variable: if all we know is that x ∈ [ x, x ], then MaxEnt leads to a uniform distribution on Full Screen [ x, x ]. Close • Example – multiple variables: different variables are Quit independently distributed.

5. Limitations of Maximum Entropy Approach • Example: simplest algorithm y = x 1 + . . . + x n . • Measurement errors: ∆ x i ∈ [ − ∆ , ∆]. • Analysis: ∆ y = ∆ x 1 + . . . + ∆ x n . Title Page • Worst case situation: ∆ y = n · ∆. ◭◭ ◮◮ • Maximum Entropy approach: due to Central Limit The- √ n ◭ ◮ √ orem, ∆ y is ≈ normal, with σ = ∆ · 3. Page 6 of 48 • Why this may be inadequate: we get ∆ ∼ √ n , but due to correlation, it is possible that ∆ = n · ∆ ∼ n ≫ √ n . Go Back Full Screen • Conclusion: using a single distribution can be very Close misleading, especially if we want guaranteed results. Quit • Examples: high-risk application areas such as space exploration or nuclear engineering.

6. Interval Arithmetic: Foundations of Interval Tech- niques • Problem: compute the range [ y, y ] = { f ( x 1 , . . . , x n ) | x 1 ∈ [ x 1 , x 1 ] , . . . , x n ∈ [ x n , x n ] } . • Interval arithmetic: for arithmetic operations f ( x 1 , x 2 ) Title Page (and for elementary functions), we have explicit formu- ◭◭ ◮◮ las for the range. ◭ ◮ • Examples: when x 1 ∈ x 1 = [ x 1 , x 1 ] and x 2 ∈ x 2 = [ x 2 , x 2 ], then: Page 7 of 48 – The range x 1 + x 2 for x 1 + x 2 is [ x 1 + x 2 , x 1 + x 2 ] . Go Back – The range x 1 − x 2 for x 1 − x 2 is [ x 1 − x 2 , x 1 − x 2 ] . Full Screen – The range x 1 · x 2 for x 1 · x 2 is [ y, y ], where Close y = min( x 1 · x 2 , x 1 · x 2 , x 1 · x 2 , x 1 · x 2 ); Quit y = max( x 1 · x 2 , x 1 · x 2 , x 1 · x 2 , x 1 · x 2 ) . • The range 1 / x 1 for 1 /x 1 is [1 /x 1 , 1 /x 1 ] (if 0 �∈ x 1 ).

7. Straightforward Interval Computations: Example • Example: f ( x ) = ( x − 2) · ( x + 2), x ∈ [1 , 2]. • How will the computer compute it? • r 1 := x − 2; • r 2 := x + 2; Title Page • r 3 := r 1 · r 2 . ◭◭ ◮◮ • Main idea: perform the same operations, but with in- ◭ ◮ tervals instead of numbers : Page 8 of 48 • r 1 := [1 , 2] − [2 , 2] = [ − 1 , 0]; Go Back • r 2 := [1 , 2] + [2 , 2] = [3 , 4]; Full Screen • r 3 := [ − 1 , 0] · [3 , 4] = [ − 4 , 0]. Close • Actual range: f ( x ) = [ − 3 , 0]. Quit • Comment: this is just a toy example, there are more efficient ways of computing an enclosure Y ⊇ y .

8. First Idea: Use of Monotonicity • Reminder: for arithmetic, we had exact ranges. • Reason: +, − , · are monotonic in each variable. • How monotonicity helps: if f ( x 1 , . . . , x n ) is (non-strictly) increasing ( f ↑ ) in each x i , then Title Page f ( x 1 , . . . , x n ) = [ f ( x 1 , . . . , x n ) , f ( x 1 , . . . , x n )] . ◭◭ ◮◮ • Similarly: if f ↑ for some x i and f ↓ for other x j ( − ). ◭ ◮ • Fact: f ↑ in x i if ∂f Page 9 of 48 ≥ 0. ∂x i Go Back • Checking monotonicity: check that the range [ r i , r i ] of Full Screen ∂f on x i has r i ≥ 0. ∂x i Close • Differentiation: by Automatic Differentiation (AD) tools. Quit • Estimating ranges of ∂f : straightforward interval comp. ∂x i

9. Monotonicity: Example • Idea: if the range [ r i , r i ] of each ∂f on x i has r i ≥ 0, ∂x i then f ( x 1 , . . . , x n ) = [ f ( x 1 , . . . , x n ) , f ( x 1 , . . . , x n )] . Title Page • Example: f ( x ) = ( x − 2) · ( x + 2), x = [1 , 2]. ◭◭ ◮◮ • Case n = 1 : if the range [ r, r ] of d f dx on x has r ≥ 0, ◭ ◮ then Page 10 of 48 f ( x ) = [ f ( x ) , f ( x )] . Go Back • AD: d f dx = 1 · ( x + 2) + ( x − 2) · 1 = 2 x . Full Screen Close • Checking: [ r, r ] = [2 , 4], with 2 ≥ 0. Quit • Result: f ([1 , 2]) = [ f (1) , f (2)] = [ − 3 , 0]. • Comparison: this is the exact range.

10. Non-Monotonic Example • Example: f ( x ) = x · (1 − x ), x ∈ [0 , 1]. • How will the computer compute it? • r 1 := 1 − x ; • r 2 := x · r 1 . Title Page • Straightforward interval computations: ◭◭ ◮◮ • r 1 := [1 , 1] − [0 , 1] = [0 , 1]; ◭ ◮ • r 2 := [0 , 1] · [0 , 1] = [0 , 1]. Page 11 of 48 • Actual range: min, max of f at x , x , or when d f dx = 0. Go Back • Here, d f Full Screen dx = 1 − 2 x = 0 for x = 0 . 5, so Close – compute f (0) = 0, f (0 . 5) = 0 . 25, and f (1) = 0 . Quit – y = min(0 , 0 . 25 , 0) = 0, y = max(0 , 0 . 25 , 0) = 0 . 25. • Resulting range: f ( x ) = [0 , 0 . 25].

11. Second Idea: Centered Form • Main idea: Intermediate Value Theorem n � ∂f f ( x 1 , . . . , x n ) = f ( � x 1 , . . . , � x n ) + ( χ ) · ( x i − � x i ) ∂x i i =1 for some χ i ∈ x i . Title Page • Corollary: f ( x 1 , . . . , x n ) ∈ Y , where ◭◭ ◮◮ n � ◭ ◮ ∂f Y = � y + ( x 1 , . . . , x n ) · [ − ∆ i , ∆ i ] . ∂x i Page 12 of 48 i =1 Go Back • Differentiation: by Automatic Differentiation (AD) tools. Full Screen • Estimating the ranges of derivatives: Close – if appropriate, by monotonicity, or Quit – by straightforward interval computations, or – by centered form (more time but more accurate).

12. Centered Form: Example • General formula: n � ∂f Y = f ( � x 1 , . . . , � x n ) + ( x 1 , . . . , x n ) · [ − ∆ i , ∆ i ] . ∂x i i =1 • Example: f ( x ) = x · (1 − x ), x = [0 , 1]. Title Page • Here, x = [ � x − ∆ , � x + ∆], with � x = 0 . 5 and ∆ = 0 . 5. ◭◭ ◮◮ x ) + d f ◭ ◮ • Case n = 1 : Y = f ( � dx ( x ) · [ − ∆ , ∆]. Page 13 of 48 • AD: d f dx = 1 · (1 − x ) + x · ( − 1) = 1 − 2 x . Go Back • Estimation: we have d f Full Screen dx ( x ) = 1 − 2 · [0 , 1] = [ − 1 , 1]. Close • Result: Y = 0 . 5 · (1 − 0 . 5) + [ − 1 , 1] · [ − 0 . 5 , 0 . 5] = Quit 0 . 25 + [ − 0 . 5 , 0 . 5] = [ − 0 . 25 , 0 . 75]. • Comparison: actual range [0 , 0 . 25], straightforward [0 , 1].

13. Third Idea: Bisection • Known: accuracy O (∆ 2 i ) of first order formula n � ∂f f ( x 1 , . . . , x n ) = f ( � x 1 , . . . , � x n ) + ( χ ) · ( x i − � x i ) . ∂x i i =1 • Idea: if the intervals are too wide, we: Title Page – split one of them in half (∆ 2 i → ∆ 2 i / 4); and ◭◭ ◮◮ – take the union of the resulting ranges. ◭ ◮ • Example: f ( x ) = x · (1 − x ), where x ∈ x = [0 , 1]. Page 14 of 48 • Split: take x ′ = [0 , 0 . 5] and x ′′ = [0 . 5 , 1]. Go Back • 1st range: 1 − 2 · x = 1 − 2 · [0 , 0 . 5] = [0 , 1], so f ↑ and Full Screen f ( x ′ ) = [ f (0) , f (0 . 5)] = [0 , 0 . 25]. Close • 2nd range: 1 − 2 · x = 1 − 2 · [0 . 5 , 1] = [ − 1 , 0], so f ↓ Quit and f ( x ′′ ) = [ f (1) , f (0 . 5)] = [0 , 0 . 25]. • Result: f ( x ′ ) ∪ f ( x ′′ ) = [0 , 0 . 25] – exact.