Code Generators for Mathematical Functions N. Brunie 1 , F. de - PowerPoint PPT Presentation

Code Generators for Mathematical Functions N. Brunie 1 , F. de Dinechin 2 , O. Kupriianova 3 , Ch. Lauter 3 1 Kalray , Grenoble, France 2 Universit´ e de Lyon, INRIA, INSA-Lyon, CITI , F-69621 Villeurbanne, France 3 Sorbonne Universit´ es, UPMC Univ Paris 06, UMR 7606, LIP6 , F-75005 Paris, France 22nd IEEE Symposium on Computer Arithmetic Lyon, France, 22-24 June 2015 O. Kupriianova (LIP6) Code generators for math functions ARITH 22, 23 June 2015 1 /30

Mathematical libraries Standard libraries (libms) elementary functions (exp , log , sin , sinh) special functions ( x y , Γ, Bessel) standard precisions (single, double, quad) Existing implementations glibc libm Intel’s MKL CRLibm by ENS Lyon AMD’s libm newlib ARM’s mathlib OpenLibm for Julia libmcr by Sun Yeppp! . . . O. Kupriianova (LIP6) Code generators for math functions ARITH 22, 23 June 2015 2 /30

One size does not fit all Current offer Several performance options (latency vs throughput) Several accuracy options (“quick and dirty”, faithful, correctly-rounded) Several portability options (generic vs AVX512) O. Kupriianova (LIP6) Code generators for math functions ARITH 22, 23 June 2015 3 /30

One size does not fit all Current offer Several performance options (latency vs throughput) Several accuracy options (“quick and dirty”, faithful, correctly-rounded) Several portability options (generic vs AVX512) Some people are still not happy More performance, less compliance degraded accuracy reduced domain Functions not from standard libm O. Kupriianova (LIP6) Code generators for math functions ARITH 22, 23 June 2015 3 /30

One size does not fit all Current offer Several performance options (latency vs throughput) Several accuracy options (“quick and dirty”, faithful, correctly-rounded) Several portability options (generic vs AVX512) Some people are still not happy More performance, less compliance degraded accuracy reduced domain Functions not from standard libm Who is going to write all these variants? O. Kupriianova (LIP6) Code generators for math functions ARITH 22, 23 June 2015 3 /30

Solution Metalibm Write tools to produce code for math functions Analogy assembly → compilers code → code generators . c f i s t a r t p r o c subq $8 , %rsp #i n c l u d e < s t d i o . h > . c f i d e f c f a o f f s e t 16 movl $52 , %r8d i n t main () { movl $37 , %ecx a , b , sum ; i n t movl $15 , %edx a = 15; movl $ . LC0 , %e s i b = 37; sum = a + b ; movl $1 , %e d i x o r l %eax , %eax p r i n t f ( ”%d + %d = %d \ n” , a , b , c a l l p r i n t f c h k sum) ; x o r l %eax , %eax return 0 ; addq $8 , %rsp } . c f i d e f c f a o f f s e t 8 r e t . c f i e n d p r o c O. Kupriianova (LIP6) Code generators for math functions ARITH 22, 23 June 2015 4 /30

Metalibm: generator use-cases open-ended code generation user input generated code black-box approximation scheme function generator back end C code specification sollya function generator code I/O precision python C11 function target accuracy code generators test range code python processor C11 libm code generation O. Kupriianova (LIP6) Code generators for math functions ARITH 22, 23 June 2015 5 /30

Outline Background in function implementations 1 Lutetia version (open-ended generation) 2 Lugdunum version (C11 function code generator) 3 Conclusion and Future Work 4 O. Kupriianova (LIP6) Code generators for math functions ARITH 22, 23 June 2015 6 /30

How to implement a function manually � � fun − f � � ≤ ¯ The task: from f on [ a , b ] get an implementation fun : ε f O. Kupriianova (LIP6) Code generators for math functions ARITH 22, 23 June 2015 7 /30

How to implement a function manually � � fun − f � � ≤ ¯ The task: from f on [ a , b ] get an implementation fun : ε f 1. Eliminating special cases: zeros, infinities, NaNs, etc. 2. Argument reduction: transform [ a , b ] to [ α, β ], a small interval 3. Polynomial approximation: minimax approximation, polynomial of low degree, Remez algorithm 4. Reconstruction O. Kupriianova (LIP6) Code generators for math functions ARITH 22, 23 June 2015 7 /30

How to implement a function manually � � fun − f � � ≤ ¯ The task: from f on [ a , b ] get an implementation fun : ε f 1. Eliminating special cases: zeros, infinities, NaNs, etc. 2. Argument reduction: transform [ a , b ] to [ α, β ], a small interval 3. Polynomial approximation: minimax approximation, polynomial of low degree, Remez algorithm 4. Reconstruction Example implement f ( x ) = e x � � � � e x = 2 x x x x = 2 E · e x − E log 2 = 2 E · e r log 2 − log 2 = 2 · 2 log 2 log 2 O. Kupriianova (LIP6) Code generators for math functions ARITH 22, 23 June 2015 7 /30

Argument reduction Based on mathematical properties: n a + b = n a · n b , sin( x + 2 π ) = sin( x ) , log( a · b ) = log( a ) + log( b ) , . . . O. Kupriianova (LIP6) Code generators for math functions ARITH 22, 23 June 2015 8 /30

Argument reduction Based on mathematical properties: n a + b = n a · n b , sin( x + 2 π ) = sin( x ) , log( a · b ) = log( a ) + log( b ) , . . . What properties do we know for erf , J 0 or an open-ended function (purely defined by a differential equation)? O. Kupriianova (LIP6) Code generators for math functions ARITH 22, 23 June 2015 8 /30

Argument reduction Based on mathematical properties: n a + b = n a · n b , sin( x + 2 π ) = sin( x ) , log( a · b ) = log( a ) + log( b ) , . . . What properties do we know for erf , J 0 or an open-ended function (purely defined by a differential equation)? When argument reduction does not work Piecewise-polynomial approximation O. Kupriianova (LIP6) Code generators for math functions ARITH 22, 23 June 2015 8 /30

Outline Background in function implementations 1 Lutetia version (open-ended generation) 2 Lugdunum version (C11 function code generator) 3 Conclusion and Future Work 4 O. Kupriianova (LIP6) Code generators for math functions ARITH 22, 23 June 2015 9 /30

Metalibm: generator use-cases open-ended code generation black-box approximation scheme function generator back end C code specification sollya function generator code I/O precision python C11 function target accuracy code generators test range user input code python processor C11 libm code generation O. Kupriianova (LIP6) Code generators for math functions ARITH 22, 23 June 2015 10 /30

Philosophy Objective #1: push-button tool to implement functions f : R → R Similar to yesterday’s talk by D. Thomas, but in software automatic argument reduction automatic polynomial approximation automatic domain splitting with user specified accuracy O. Kupriianova (LIP6) Code generators for math functions ARITH 22, 23 June 2015 11 /30

Philosophy Objective #1: push-button tool to implement functions f : R → R Similar to yesterday’s talk by D. Thomas, but in software automatic argument reduction automatic polynomial approximation automatic domain splitting with user specified accuracy Objective #2: black-box functions Open-ended means: no function dictionaries specify the function by an expression (composite functions) but not only: all we need is code that evaluates the function and its first derivatives with arbitrary accuracy O. Kupriianova (LIP6) Code generators for math functions ARITH 22, 23 June 2015 11 /30

Black-box function generator Metalibm Lutetia parameters implementation.c O. Kupriianova (LIP6) Code generators for math functions ARITH 22, 23 June 2015 12 /30

Black-box function generator Metalibm Lutetia parameters implementation.c 1 function 2 domain final accuracy 3 max poly degree table size 1 - Properties detection 2 - Domain splitting 3 - Approximation O. Kupriianova (LIP6) Code generators for math functions ARITH 22, 23 June 2015 13 /30

Exponential function detection Generation hypothesis f ( x ) is of type β x , unknown β O. Kupriianova (LIP6) Code generators for math functions ARITH 22, 23 June 2015 14 /30

Exponential function detection Generation hypothesis f ( x ) is of type β x , unknown β Finding the base � � ln( f ( ξ )) β = exp , for some ξ ∈ [ a , b ] ξ O. Kupriianova (LIP6) Code generators for math functions ARITH 22, 23 June 2015 14 /30

Exponential function detection Generation hypothesis f ( x ) is of type β x , unknown β Finding the base � � ln( f ( ξ )) β = exp , for some ξ ∈ [ a , b ] ξ Decision of acceptance [ a , b ] � � � β x ε = ˜ f ( x ) − 1 should be small � � � ∞ O. Kupriianova (LIP6) Code generators for math functions ARITH 22, 23 June 2015 14 /30

Black-box function generator Metalibm Lutetia parameters implementation.c 1 function 2 domain final accuracy 3 max poly degree table size 1 - Properties detection 2 - Domain splitting 3 - Approximation O. Kupriianova (LIP6) Code generators for math functions ARITH 22, 23 June 2015 15 /30

Domain splitting hints General procedures Naive: choose some large k Hierarchical: split into 2 k subdomains Successive powers of two Function-adapted procedures Bisection Optimized bisection O. Kupriianova (LIP6) Code generators for math functions ARITH 22, 23 June 2015 16 /30