Exact computations with an arithmetic known to be approximate - PowerPoint PPT Presentation

Exact computations with an arithmetic known to be approximate MaGiX@LiX conference – 2011 Jean-Michel Muller CNRS - Laboratoire LIP (CNRS-INRIA-ENS Lyon-Université de Lyon) http://perso.ens-lyon.fr/jean-michel.muller/ J.-M. Muller Exact computations with an arithmetic. . . sept. 2011 1 / 55

Introduction Floating-Point Arithmetic Floating-Point Arithmetic bad reputation ; used everywhere in scientific calculation ; “scientific notation” of numbers : 6 . 02214179 × 10 23 The number 6 . 02214179 is the significand (or mantissa ), and the number 23 is the exponent. generalization to radix β : x = m x · β e x , where m x is represented in radix β . Almost always, β is 2 or 10 ; J.-M. Muller Exact computations with an arithmetic. . . sept. 2011 2 / 55

Introduction Floating-Point Arithmetic Floating-Point Arithmetic bad reputation ; used everywhere in scientific calculation ; “scientific notation” of numbers : 6 . 02214179 × 10 23 The number 6 . 02214179 is the significand (or mantissa ), and the number 23 is the exponent. generalization to radix β : x = m x · β e x , where m x is represented in radix β . Almost always, β is 2 or 10 ; But there is more to say about this. . . later J.-M. Muller Exact computations with an arithmetic. . . sept. 2011 2 / 55

Introduction Desirable properties Desirable properties Speed : tomorrow’s weather must be computed in less than 24 hours ; Accuracy, Range ; “Size” : silicon area and/or code size ; Power consumption ; Portability : the programs we write on a given system must run on different systems without requiring huge modifications ; Easiness of implementation and use : If a given arithmetic is too arcane, nobody will use it. J.-M. Muller Exact computations with an arithmetic. . . sept. 2011 3 / 55

Introduction Famous failures Some can do a very poor job. . . 1994 : Pentium 1 division bug : 8391667 / 12582905 gave 0 . 666869 · · · instead of 0 . 666910 · · · ; J.-M. Muller Exact computations with an arithmetic. . . sept. 2011 4 / 55

Introduction Famous failures Some can do a very poor job. . . 1994 : Pentium 1 division bug : 8391667 / 12582905 gave 0 . 666869 · · · instead of 0 . 666910 · · · ; Maple version 6.0. Enter 214748364810, you get 10. Note that 2147483648 = 2 31 ; J.-M. Muller Exact computations with an arithmetic. . . sept. 2011 4 / 55

Introduction Famous failures Some can do a very poor job. . . 1994 : Pentium 1 division bug : 8391667 / 12582905 gave 0 . 666869 · · · instead of 0 . 666910 · · · ; Maple version 6.0. Enter 214748364810, you get 10. Note that 2147483648 = 2 31 ; Excel’2007 (first releases), compute 65535 − 2 − 37 , you get 100000 ; J.-M. Muller Exact computations with an arithmetic. . . sept. 2011 4 / 55

Introduction Famous failures Some can do a very poor job. . . 1994 : Pentium 1 division bug : 8391667 / 12582905 gave 0 . 666869 · · · instead of 0 . 666910 · · · ; Maple version 6.0. Enter 214748364810, you get 10. Note that 2147483648 = 2 31 ; Excel’2007 (first releases), compute 65535 − 2 − 37 , you get 100000 ; November 1998, USS Yorktown warship, somebody erroneously entered a «zero» on a keyboard → division by 0 → series of errors → the propulsion system stopped. J.-M. Muller Exact computations with an arithmetic. . . sept. 2011 4 / 55

Introduction Famous failures Some strange things Setun Computer, Moscow University, 1958. 50 copies ; J.-M. Muller Exact computations with an arithmetic. . . sept. 2011 5 / 55

Introduction Famous failures Some strange things Setun Computer, Moscow University, 1958. 50 copies ; radix 3 and digits − 1, 0 and 1. Numbers represented using 18 « trits » ; J.-M. Muller Exact computations with an arithmetic. . . sept. 2011 5 / 55

Introduction Famous failures Some strange things Setun Computer, Moscow University, 1958. 50 copies ; radix 3 and digits − 1, 0 and 1. Numbers represented using 18 « trits » ; idea : radix β , n digits, you want to represent around M different numbers. “Cost” : β × n . J.-M. Muller Exact computations with an arithmetic. . . sept. 2011 5 / 55

Introduction Famous failures Some strange things idea : radix β , n digits, you want to represent around M different numbers. “Cost” : β × n . → punched card area ; J.-M. Muller Exact computations with an arithmetic. . . sept. 2011 6 / 55

Introduction Famous failures Some strange things idea : radix β , n digits, you want to represent around M different numbers. “Cost” : β × n . → punched card area ; if we wish to represent M numbers, minimize β × n knowing that β n ≥ M . J.-M. Muller Exact computations with an arithmetic. . . sept. 2011 6 / 55

Introduction Famous failures Some strange things idea : radix β , n digits, you want to represent around M different numbers. “Cost” : β × n . → punched card area ; if we wish to represent M numbers, minimize β × n knowing that β n ≥ M . With real variables β = e = 2 . 718 . . . ≈ 3. . . what is the “best” (integral) radix ? J.-M. Muller Exact computations with an arithmetic. . . sept. 2011 6 / 55

Introduction Famous failures Some strange things idea : radix β , n digits, you want to represent around M different numbers. “Cost” : β × n . → punched card area ; if we wish to represent M numbers, minimize β × n knowing that β n ≥ M . With real variables β = e = 2 . 718 . . . ≈ 3. . . what is the “best” (integral) radix ? as soon as : 5 ( 2 / ln ( 2 )) − ( 3 / ln ( 3 )) ≈ 1 . 09 × 10 14 M ≥ e it is always 3 J.-M. Muller Exact computations with an arithmetic. . . sept. 2011 6 / 55

Introduction Famous failures Yes, but. . . Building circuits with three-valued logic turned out to be very difficult. . . J.-M. Muller Exact computations with an arithmetic. . . sept. 2011 7 / 55

Introduction Famous failures Yes, but. . . Building circuits with three-valued logic turned out to be very difficult. . . . . . so that in practice, each “trit” was represented by two bits. J.-M. Muller Exact computations with an arithmetic. . . sept. 2011 7 / 55

Introduction Definition Floating-Point System Parameters :  radix (or base) β ≥ 2 (will be 2 in this presentation)  precision p ≥ 1  extremal exponents e min , e max , A finite FP number x is represented by 2 integers : integral significand : M , | M | ≤ β p − 1 ; exponent e , e min ≤ e ≤ e max . such that x = M × β e + 1 − p with | M | largest under these constraints ( → | M | ≥ β p − 1 , unless e = e min ). (Real) significand of x : the number m = M × β 1 − p , so that x = m × β e . J.-M. Muller Exact computations with an arithmetic. . . sept. 2011 8 / 55

Introduction Definition Normal and subnormal numbers normal number : of absolute value ≥ β e min . The absolute value of its integral significand is ≥ β p − 1 ; subnormal number : of absolute value < β e min . The absolute value of its integral significand is < β p − 1 . normality/subnormality encoded in the exponent. Radix 2 : the leftmost bit of the significand of a normal number is a “1” → no need to store it (implicit 1 convention). J.-M. Muller Exact computations with an arithmetic. . . sept. 2011 9 / 55

Introduction Definition Subnormal numbers difficult to implement efficiently, but. . . a − b a b β e min β e min + 1 β e min + 2 0 a − b a b β e min β e min + 1 β e min + 2 0 a � = b equivalent to “computed a − b � = 0”. J.-M. Muller Exact computations with an arithmetic. . . sept. 2011 10 / 55

Introduction Definition IEEE-754 Standard for FP Arithmetic (1985 and 2008) put an end to a mess (no portability, variable quality) ; leader : W. Kahan (father of the arithmetic of the HP35 and the Intel 8087) ; formats ; specification of operations and conversions ; exception handling (max+1, 1/0, √− 2, 0/0, etc.) ; new version of the standard : August 2008. J.-M. Muller Exact computations with an arithmetic. . . sept. 2011 11 / 55

Introduction Correct rounding Correct rounding Definition 1 (Correct rounding) The user chooses a rounding function among : round toward −∞ : RD ( x ) is the largest FP number ≤ x ; round toward + ∞ : RU ( x ) is the smallest FP number ≥ x ; round toward zero : RZ ( x ) is equal to RD ( x ) if x ≥ 0, and to RU ( x ) if x ≤ 0 ; round to nearest : RN ( x ) = FP number closest to x . If exactly halfway between two consecutive FP numbers : the one whose integral significand is even (default mode) Correctly rounded operation : returns what we would get by infinitely precise operation followed by rounding. J.-M. Muller Exact computations with an arithmetic. . . sept. 2011 12 / 55

Introduction Correct rounding Correct rounding IEEE-754 (1985) : Correct rounding for + , − , × , ÷ , √ and some conversions. Advantages : if the result of an operation is exactly representable, we get it ; if we just use the 4 arith. operations and √ , deterministic arithmetic : one can elaborate algorithms and proofs that use the specifications ; accuracy and portability are improved ; playing with rounding towards + ∞ and −∞ → certain lower and upper bounds : interval arithmetic. FP arithmetic becomes a structure in itself, that can be studied. J.-M. Muller Exact computations with an arithmetic. . . sept. 2011 13 / 55